A service processing method based on efficient analysis

Abstract

The present invention discloses a service processing method based on efficient analysis, including: performing traffic identification and split conversion processing on an Internet service processing system; performing echo response control on the Internet service processing system; performing analysis control management on the Internet service processing system; performing investigation transmission processing on the Internet service processing system; the performing traffic identification and split conversion processing on the Internet service processing system includes: splitting traffic from an Ethernet data packet according to the data packet header part, after stripping the data to complete the data traffic identification processing, controlling the transmission processing of the data stream through a state machine, and the three states of the state machine are header processing, data forwarding, and data discarding. The data in the header processing state includes Ethernet header information, and when processing, the type field in the Ethernet header will be obtained first. The present invention has the characteristics of intelligent control management and high efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of service processing, and particularly to a service processing method based on efficient analysis. Background Art

[0002] With the increasing development of Internet technology, the amount of data generated by individuals and enterprises is constantly increasing, and the difficulty of data processing services is also increasing. It not only requires the ability to process large-scale data, but also needs to minimize data response latency. In order to improve the ability to process large batches of data flow services and improve data transmission efficiency, stream processing technology has emerged. Both stream processing and batch processing are common means of processing data. Compared with stream processing, batch processing usually processes non-online data, cold data, and a large amount of single-process data, and the processing speed is slower than that of stream processing. Stream processing targets on-line and real-time generated data, with a very small amount of data processed at a time, but lower processing latency. The large amount of data brought by stream processing technology also brings severe computational pressure to the data center. The data center often cannot generate the required computational results in time, thus seriously affecting Internet data processing services. Therefore, it is necessary to design an intelligent control management and efficient service processing method based on efficient analysis. Summary of the Invention

[0003] The purpose of the present invention is to provide a service processing method based on efficient analysis to solve the problems raised in the above background art.

[0004] To solve the above technical problems, the present invention provides the following technical solutions: A service processing method based on efficient analysis, including:

[0005] Perform traffic identification and split conversion processing on the Internet service processing system;

[0006] Perform echo response control on the Internet service processing system;

[0007] Perform analysis control management on the Internet service processing system;

[0008] Perform investigation transmission processing on the Internet service processing system;

[0009] The performing traffic identification and split conversion processing on the Internet service processing system includes:

[0010] Split the traffic from the Ethernet packet according to the data header part. After stripping the data and completing the data traffic identification process, control the transmission process of the data stream through a state machine. The three states of the state machine are header processing, data forwarding, and data discarding. The data in the header processing state includes Ethernet header information. When processing, the type field in the Ethernet header will be obtained first. If the type field is ARP, the tdest signal will be set to 0. If the type field is IPV4 and the IP version number field is 4, the protocol type will be judged. The tdest signal of the ICMP type will be set to 1, the tdest of the TCP type will be set to 2, and UDP will be set to 3;

[0011] During the traffic identification process, after entering the header processing state, first judge whether the packet type is ARP. Since ARP requires the MAC address information in the Ethernet header during processing, the packet in ARP format needs to retain the Ethernet header for forwarding and directly enter the data forwarding state. Packets of other types need to remove the Ethernet header. When processing, directly store the data after the Ethernet header into the forwarding variable and consider it as data with a set length offset. At the end of the header processing state, it is necessary to judge whether the end of the packet has been reached through the last signal of the packet. If the end of the packet has not been reached, the next state is the data elimination state, otherwise enter the header processing state again.

[0012] According to the above technical solution, the echo response control of the Internet service processing system includes:

[0013] When reading the packet in the Internet service processing system, read the destination IP address from the IP field, read the ICMP type field and ICMP code field from the ICMP header, and analyze to obtain the ICMP packet checksum. At the same time, add an 8-bit reverse rearrangement operation for the IP header.

[0014] According to the above technical solution, the analysis control management of the Internet service processing system includes:

[0015] Store the data table of the Internet service processing system inside the FPGA, allocate internal DDR4 space for the data table, and further configure the XDMA memory access mode. Read the data in the internal DDR4 space of the FPGA through the PCIE channel, and establish a translation lookaside buffer (TLB) in the FPGA. Use the last 5 bits of the port number to construct a data table with a length of 32 to store the table ID information of the Socket table. Therefore, after the configuration is completed, the Socket table entry can be directly located through the port number, saving the loop traversal operation.

[0016] According to the above technical solution, the analysis control management of the Internet service processing system further includes:

[0017] Design a custom data segment format within UDP. In addition to the original UDP header, further add fields such as packet sequence number, number of sub - packets, current sub - packet sequence number, and timestamp. When sending data, for data packets exceeding 9000 bytes, send them in sub - packets. When sending sub - packets, set the packet sequence number to the same value, and the sub - packet sequence number increases sequentially. Further complete multi - packet caching by using a cache table;

[0018] The control channel is the interaction channel between the NodeManager structure of the node host and the control information of the master node, completing the information transmission between the master and slave nodes in the system.

[0019] According to the above - mentioned technical solution, the survey transmission processing of the Internet service processing system includes:

[0020] During the user's use of the Internet service processing system, issue a questionnaire to the user to collect and survey the user's opinion information;

[0021] Transmit the user - collected survey information to the workbench to make the subsequent optimization of the system more in line with the user's needs.

[0022] According to the above - mentioned technical solution, a service processing system based on efficient analysis includes:

[0023] An identification processing module for performing identification processing on the Internet service processing system;

[0024] An analysis and management module for performing analysis, optimization, and management of the Internet service processing system;

[0025] A survey transmission module for performing survey transmission of user opinion information.

[0026] According to the above - mentioned technical solution, the identification processing module includes:

[0027] A traffic identification module for performing identification processing on the number of users and traffic of the Internet service processing system;

[0028] A splitting and conversion module for controlling the splitting and conversion of data packets;

[0029] An echo response module for managing the echo response of data packet reading.

[0030] According to the above - mentioned technical solution, the analysis and management module includes:

[0031] A configuration processing module for performing FPGA configuration processing on the Internet service processing system;

[0032] A multi - packet caching module for controlling multi - packet caching of the Internet service processing system;

[0033] A transmission control module for performing transmission control management of data information between nodes.

[0034] According to the above technical solution, the investigation transmission module includes:

[0035] A service investigation module for conducting service investigations on user opinion information;

[0036] A transmission processing module for performing transmission control processing of data information.

[0037] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: By providing an identification processing module, an analysis management module, and an investigation transmission module, the present invention makes data transmission service processing more efficient, effectively improves the accuracy and stability of data service processing, and at the same time can improve the system data analysis processing efficiency, avoid the generation of analysis processing errors, and can be configured in the form of external memory access through PCIE while meeting the high-speed data reading requirements of the system, effectively improving the operation service quality of the system. Description of the Drawings

[0038] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0039] Figure 1 is a flowchart of a service processing method based on efficient analysis provided by Embodiment 1 of the present invention;

[0040] Figure 2 is a module composition diagram of a service processing system based on efficient analysis provided by Embodiment 2 of the present invention. Detailed Embodiments

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Embodiment 1: Figure 1 is a flowchart of a service processing method based on efficient analysis provided by Embodiment 1 of the present invention. This embodiment can be applied to an Internet service processing system. This method can be executed by a service processing system based on efficient analysis provided by the embodiments of the present invention. The system consists of multiple software and hardware modules, such as Figure 1 shown, and the method specifically includes the following steps:

[0043] S101. Perform traffic identification and split conversion processing on the Internet service processing system;

[0044] Exemplarily, in the embodiment of the present invention, traffic is segmented from the Ethernet data packet according to the data packet header. After stripping the data and completing the data traffic identification processing, the transmission processing of the data stream is controlled by a state machine. The three states of the state machine are header processing, data forwarding, and data discarding. The data in the header processing state includes Ethernet header information. When processing, the type field in the Ethernet header is first obtained. If the type field is ARP, the tdest signal is set to 0. If the type field is IPV4 and the IP version number field is 4, the protocol type is judged. The tdest signal of the ICMP type is set to 1, the tdest of the TCP type is set to 2, and UDP is set to 3; in this step, tdest represents data in the ARP, ICMP, TCP, and UDP formats from 0 to 3. Except for the cases where the tdest signal is set in this step, the data packets in other cases are not forwarded. Here, a boolean type variable, called forwarding judgment, is used for control. At the end of the header processing state, the conversion to the next state is performed. Through the forwarding judgment variable, if the variable value meets the set requirements, it means that this type of data packet needs to be further processed and will enter the data forwarding state, otherwise it will enter the data discarding state. In the data forwarding state, the subsequent data will be stored in the forwarding variable, including the byte information in the data packet, the keep field, and the tdest state obtained in the header processing state. The data forwarding state will repeatedly obtain and forward the data until the last signal of the data packet is obtained. After the data forwarding state ends, it will re-enter the header processing state. In the data discarding state, a null operation will be performed until the last signal of the data packet is obtained. After this state ends, it will enter the header processing state.

[0045] During the traffic recognition process, after entering the header processing state, first determine whether the packet type is ARP. Since ARP requires the MAC address information in the Ethernet header during processing, packets in ARP format need to retain the Ethernet header for forwarding and directly enter the data forwarding state. Packets of other types need to remove the Ethernet header. When processing, directly store the data after the Ethernet header into the forwarding variable and consider it as the data having an offset of the set length. At the end of the header processing state, determine whether the end of the packet has been reached through the last signal of the packet. If the end of the packet has not been reached, the next state is the data elimination state; otherwise, enter the header processing state again. In this step, in the data elimination state, since the Ethernet header is removed, it is equivalent to a storage data offset. Therefore, a part of the data in each group of transmitted data needs to be retained for the next transmission group. Therefore, the fields sent out from the data elimination state include the second half of the data in the previous group and the first half of the data in this group. Repeat this process until the end of the packet is reached, that is, the last signal is true. After the last group of data arrives, it is necessary to determine whether the second half of the data in this group has been transmitted. If it has not been transmitted, enter the extra data state and send the remaining data to make the data transmission service processing more efficient and effectively improve the accuracy and stability of the data service processing.

[0046] S102. Perform echo reply control on the Internet service processing system;

[0047] Exemplarily, in the embodiment of the present invention, when reading a packet in the Internet service processing system, read the destination IP address from the IP field, read the ICMP type field and the ICMP code field from the ICMP header, and analyze to obtain the ICMP packet checksum. At the same time, add an 8-bit reverse rearrangement operation for each group of the IP header. In this step, after the packet reading is completed, it will enter the state evaluation stage. First, perform an echo judgment. The judgment conditions include that the previously obtained IP address field is equal to the IP address on the FPGA side, the ICMP type field is an echo request and the ICMP code field is 0, and the checksum calculation is 0. If the judgment conditions are met, enter the header sending stage; otherwise, enter the next judgment. If the end of the packet is reached, that is, the last field is true, enter the packet acquisition stage. If the end of the packet has not been reached, discard the data. Through this step, the data analysis and processing efficiency of the system is effectively improved, and the generation of analysis and processing errors is avoided.

[0048] S103. Perform analysis control management on the Internet service processing system;

[0049] Exemplarily, in an embodiment of the present invention, the data table of the Internet service processing system is stored inside the FPGA, an internal DDR4 space is allocated for the data table, and the XDMA memory access mode is further configured. The data in the internal DDR4 space of the FPGA is read through the PCIE channel, and a fast table is established in the FPGA. A data table with a length of 32 is constructed using the last 5 bits of the port number to store the table ID information of the Socket table. Therefore, after the configuration is completed, the Socket table entry can be directly located through the port number, eliminating the loop traversal operation. In this step, the DDR4 is the internal high-speed storage space of the FPGA, which has a high data reading rate and operating frequency, and can be accessed by each function of the system. It can be configured as an external memory access form through the PCIE while meeting the high-speed data reading requirements of the system, effectively improving the operating service quality of the system.

[0050] Design a custom data segment format within UDP. In addition to the original UDP header, fields such as packet sequence number, number of sub-packets, current sub-packet sequence number, and timestamp are further added. When sending data, for data packets exceeding 9000 bytes, they are sent in sub-packets. When sending sub-packets, the packet sequence number is set to the same value, and the sub-packet sequence numbers increase sequentially. Multi-packet caching is further completed by using a cache table. In this step, the cache table includes fields such as sequence number, timestamp, bitmap structure, and data area. The size of the cache table structure is 30, and the sequence number is set to the modulo operation of the packet sequence number by 30. Since a data packet can contain at most 5 sub-packets, the size of the cache data area is also set to 5. The bitmap structure uses the exclusive OR operation between the current bitmap state and the data packet bitmap value to determine whether the caching is completed. For example, if a data packet has three sub-packets, the data packet bitmap value is 11100, and the first and third data packets are currently cached, then the bitmap state is 10100. The value of the exclusive OR operation of the two values is not 0, indicating that the current caching is not completed. Through this step, the data caching processing of the system can be made more efficient and accurate, avoiding the out-of-order phenomenon that traditional UDP data packets have when being sent, and the need to rearrange the data packets according to the packet sequence number, which may lead to a situation of low service processing efficiency and chaos.

[0051] The control channel is the information interaction channel between the NodeManager structure of the slave node host and the control information of the master node, completing the information transmission between the master and slave nodes in the system. In this step, the transmitted control information includes the load degree of the FPGA computing resources, calculation task arrival and completion notifications, heartbeat, and fault tolerance mechanisms, etc., while the data channel is the transmission channel for computing data.

[0052] S104. Conduct the investigation transmission processing of the Internet service processing system;

[0053] Exemplarily, in the embodiments of the present invention, during the process of a user using an Internet service processing system, a questionnaire is issued to the user to collect and survey the user's opinion information;

[0054] The user's collected survey information is transmitted to the workbench, making the subsequent optimization of the system more in line with the user's needs.

[0055] Embodiment 2: Embodiment 2 of the present invention provides a service processing system based on efficient analysis. Figure 2 As shown in the schematic diagram of the module composition of a service processing system based on efficient analysis provided for this Embodiment 2, Figure 2 as shown, the system includes:

[0056] An identification processing module, used for the identification processing of the Internet service processing system;

[0057] An analysis and management module, used for the analysis, optimization, and management of the Internet service processing system;

[0058] A survey transmission module, used for the survey and transmission of the user's opinion information.

[0059] In some embodiments of the present invention, the identification processing module includes:

[0060] A traffic identification module, used for the identification processing of the number of users and traffic of the Internet service processing system;

[0061] A splitting and conversion module, used for the splitting and conversion control of data packets;

[0062] An echo response module, used for the echo response management of data packet reading.

[0063] In some embodiments of the present invention, the analysis and management module includes:

[0064] A configuration processing module, used for the FPGA configuration processing of the Internet service processing system;

[0065] A multi-packet caching module, used for the multi-packet caching control of the Internet service processing system;

[0066] A transmission control module, used for the transmission control and management of data information between nodes.

[0067] In some embodiments of the present invention, the survey transmission module includes:

[0068] A service survey module, used for the service survey of the user's opinion information;

[0069] A transmission processing module, used for the transmission control processing of data information.

[0070] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0071] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A service processing method based on efficient analysis, characterized in that: Including: Performing traffic identification and split conversion processing on the Internet service processing system; Performing echo response control on the Internet service processing system; Performing analysis control management on the Internet service processing system; Performing investigation transmission processing on the Internet service processing system; The performing of traffic identification and split conversion processing on the Internet service processing system includes: From the Ethernet packet, traffic is segmented according to the data packet header. After stripping the data to complete the data traffic identification processing, the transmission processing of the data stream is controlled by a state machine. The three states of the state machine are header processing, data forwarding, and data discarding. The data in the header processing state includes Ethernet header information. When processing, the type field in the Ethernet header is first obtained. If the type field is ARP, the tdest signal is set to 0. If the type field is IPV4 and the IP version number field is 4, the protocol type is judged, the tdest signal of the ICMP type is set to 1, the tdest of the TCP type is set to 2, and UDP is set to 3; During the traffic identification processing, after entering the header processing state, it is first judged whether the packet type is ARP. Since ARP requires the MAC address information in the Ethernet header during processing, the ARP format packet needs to retain the Ethernet header for forwarding and directly enter the data forwarding state. Other types of packets need to remove the Ethernet header. When processing, the data after the Ethernet header is directly stored in the forwarding variable and is regarded as the data having a set length offset. At the end of the header processing state, it is judged whether the end of the packet is reached through the last signal of the packet. If the end of the packet is not reached, the next state is the data discarding state, otherwise, it re-enters the header processing state.

2. The service processing method based on efficient analysis according to claim 1, characterized in that: The performing of echo response control on the Internet service processing system includes: When reading the packet in the Internet service processing system, the destination IP address is read from the IP field, the ICMP type field and the ICMP code field are read from the ICMP header, and the ICMP packet checksum is analyzed. At the same time, an 8-bit reverse rearrangement operation of the IP header is added.

3. A service processing method based on efficient analysis according to claim 1, characterized in that: The performing of analysis control management on the Internet service processing system includes: Storing the data table of the Internet service processing system inside the FPGA, allocating internal DDR4 space for the data table, and further configuring the XDMA memory access mode. A translation lookaside buffer (TLB) is established in the FPGA, and the last 5 bits of the port number are used to construct the data table to store the table ID information of the Socket table, so as to locate the Socket table entry through the port number.

4. A service processing method based on efficient analysis according to claim 1, characterized in that: The performing of analysis control management on the Internet service processing system further includes: Designing a custom data segment format inside UDP. In addition to the original UDP header, packet sequence number, sub-packet quantity, current sub-packet sequence number, and timestamp fields are further added. When sending data, for packets exceeding 9000 bytes, packet splitting is adopted for sending. When sending packets in split, the packet sequence number is set to the same value, the sub-packet sequence number increases sequentially, and multi-packet caching is further completed by using a cache table; The control channel is an interaction channel for the control information between the NodeManager structure of the node host and the master node, which completes the information transmission between the master and slave nodes in the system.

5. A service processing method based on efficient analysis according to claim 1, characterized in that: The survey transmission processing of the Internet service processing system includes: During the user's use of the Internet service processing system, a questionnaire is published to the user to collect and survey the user's opinion information; The user's collected survey information is transmitted to the workbench to make the subsequent optimization of the system more in line with the user's needs.

Images