Data processing device, data processing method, and program

Abstract

This data processing device (10) comprises a process executing unit (14) that receives a packet from a reception unit (11), receives a timestamp from a timestamp unit (12), receives flow information from a packet identifying unit (13), and executes processing based on the received packet, timestamp, and flow information. The respective processing times of the reception processing by the reception unit, timestamp processing by the timestamp unit, and packet identification processing by the packet identifying unit are constant regardless of the contents of the data to be processed. The process executing unit starts processing using at least one of a packet and a timestamp for the packet before the flow information for the packet is input thereto.

Description

[Technical field]

[0001] The present invention relates to a data processing device, a data processing method, and a program. [Background technology]

[0002] In order to grasp the communication status in a network, it is common to acquire packets flowing through the network and their information. Conventionally, packets on a network generally have a unique field structure in the design concept, but as network virtualization progresses and multiple networks are configured on a single physical configuration, the field structure of packets on a network is no longer uniquely determined. In such a situation, a communication data identification device is required to be able to flexibly select an identification target for the field structure of a packet. To solve this problem, a comparison and matching method using a TCAM (Ternary Content Addressable Memory) is generally used (for example, Non-Patent Document 1). However, since TCAM is implemented as a dedicated chip, there is a problem that it can only be introduced by an expensive dedicated device. In response to this, for hardware and software that are not dedicated chips, a design method using a high-level language such as C or JAVA that can easily accommodate new fields with few man-hours is used. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Catalyst - CAM / TCAM · [Searched on December 1, 2021], Internet<https: / / www.infraexpert.com / study / catalyst21.html> Summary of the Invention [Problem to be solved by the invention]

[0004] The high-level language is a procedural description language, and describes the processing that is performed sequentially. FIG. 9 shows a data processing device 910 as a comparative example that can be considered as a device that employs sequential processing. In this comparative example, the processing execution unit 914 starts processing after all the output data from the receiving unit 911, the time stamp unit 912, and the packet identification unit 913 are stored in the buffer 919. Therefore, the processing execution unit 914 needs to wait until all the output data (processing results) from each unit 911 to 913 are available, and the operation efficiency of the processing execution unit is poor. Such a problem can also be said to data processing devices in general other than the data processing device 910.

[0005] An object of the present invention is to improve the operation efficiency of a processing execution unit. [Means for solving the problem]

[0006] In order to solve the above problems, the data processing device of the present invention comprises a receiving unit that executes a receiving process to receive and output data, a first processing unit that executes a first processing on the data output from the receiving unit and outputs a first processing result of the first processing, a second processing unit that executes a second processing on the data after the first processing and outputs a second processing result of the second processing, and a processing execution unit that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and executes processing based on the received data, the first processing result, and the second processing result, wherein the processing time of at least the first processing and the second processing among the receiving processing, the first processing, and the second processing is constant regardless of the content of the data to be processed, and the processing execution unit starts processing using at least one of the data and the first processing result for the data before receiving the second processing result.

[0007] In order to solve the above problems, the data processing method of the present invention includes a receiving step of executing a receiving process to receive and output data, a first processing step of executing a first processing on the data output in the receiving step and outputting a first processing result of the first processing, a second processing step of executing a second processing on the data after the first processing and outputting a second processing result of the second processing, and a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result, wherein the processing time of at least the first processing and the second processing among the receiving process, the first processing, and the second processing is constant regardless of the content of the data to be processed, and in the processing execution step, processing using at least one of the data and the first processing result for the data is started before the second processing result is received.

[0008] In order to solve the above problem, the computer of the present invention has the computer execute a receiving step of executing a receiving process to receive and output data, a first processing step of executing a first processing on the data output in the receiving step and outputting a first processing result of the first processing, a second processing step of executing a second processing on the data after the first processing and outputting a second processing result of the second processing, and a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result, wherein the processing time of at least the first processing and the second processing among the receiving process, the first processing, and the second processing is constant regardless of the content of the data to be processed, and in the processing execution step, processing using at least one of the data and the first processing result for the data is started before the second processing result is received. Effect of the Invention

[0009] According to the present invention, the operation efficiency of the processing execution unit is improved. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing the configuration of a data processing device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a data structure of a packet. [Diagram 3] FIG. 3 is a diagram illustrating an example of the structure of the rule table. [Figure 4] FIG. 4 is a diagram for explaining a method for making the processing time constant. [Diagram 5] FIG. 5 is a diagram for explaining a method for making the processing time constant. [Figure 6] FIG. 6 is a timing chart showing the operation of the data processing device. [Figure 7] FIG. 7 is a timing chart showing the operation of the data processing device. [Figure 8] FIG. 8 is a diagram showing the configuration of the data processing device of FIG. 1 implemented by a computer. [Figure 9] FIG. 1 is a configuration diagram of a data processing device according to a comparative example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] 1, a data processing device 10 according to this embodiment includes a receiving unit 11, a time stamp unit 12 configured as a first processing unit, a packet identification unit 13 configured as a second processing unit, a processing execution unit 14, an output unit 15, and a storage unit 19. The data processing device 10 thus configured is configured as a traffic monitoring device that monitors traffic in a communication network NW for each flow.

[0013] At least a part of each of the above units 11 to 14 is composed of a logic circuit written in, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). At least a part of each of the units 11 to 14 may be composed of a processor such as a CPU (Central Processing Unit) that executes a program. The storage unit 19 is made of an appropriate non-volatile storage device such as a flash memory or an SSD (Solid State Drive). The storage unit 19 may be composed of at least a part of a memory such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Each of the units 11 to 14 may be composed of a memory that temporarily holds data being processed, etc.

[0014] The receiver 11 receives packets (more specifically, mirroring packets) flowing through the communication network NW and executes a receiving process of outputting the packets to the time stamp unit 12 and the process execution unit 14. As shown in FIG. 2, a packet is composed of data (also called a payload) and a header arranged before the data. Each piece of data constituting the header is assigned a meaning for each number of bits. This bit-separated data with a given meaning is called a header field value, and a group of packets with matching header field values ​​is called a flow. Therefore, the header field value is also information for identifying a flow. The header field value may include each IP (Internet Protocol) address indicating the source and destination of the L3 header. A flow may be, for example, a set of packets with a common header field value indicating at least one of the destination, source, and communication protocol.

[0015] Returning to Fig. 1, the time stamp unit 12 executes time stamp processing by referring to a real time clock or the like, generating a time stamp for the packet output from the receiving unit 11, and outputting the time stamp to the processing execution unit 14. The time stamp unit 12 outputs the packet from the receiving unit 11 to the packet identification unit 13. At this time, the packet may have been given a time stamp.

[0016] The packet identification unit 13 executes a packet identification process to identify a flow to which a packet from the time stamp unit 12 in which a time stamp is generated belongs, and to output a flow ID, which is flow information specifying the identified flow. The packet identification unit 13 includes a header analysis unit 13A, a hash calculation unit 13B, and a rule matching unit 13C, and each of these units 13A to 13B executes the following processes to execute the packet identification process.

[0017] A packet input from the time stamp unit 12 to the packet identification unit 13 is first input to the header analysis unit 13A. The header analysis unit 13A executes an analysis process to analyze the header of the input packet and extract a header field value from the header. The extracted header field value is input to the hash calculation unit 13B.

[0018] The hash calculation unit 13B executes a hash calculation process to hash the input header field value, and outputs the hashed header field value together with the timestamp to the rule matching unit 13C.

[0019] The rule matching unit 13C executes a matching process to compare the input header field value (hashed header field value) with a predetermined rule that specifies packets belonging to a flow to be monitored and is registered in the rule table 19A stored in the storage unit 19. As shown in FIG. 3, the rules registered in the rule table 19A are registered for each flow, and one rule includes the header field value (hashed header field value) of the flow and a flow ID associated with the header field value. The flow ID may be a rule ID that identifies the rule. In the matching process, the rule matching unit 13C refers to each rule in the rule table 19A and acquires a flow ID that corresponds to the input header field value. The rule matching unit 13C outputs the acquired flow ID to the process execution unit 14.

[0020] The processing execution unit 14 receives a packet from the receiving unit 11, a timestamp from the timestamp unit 12, and a flow ID from the packet identification unit 13, and executes processing (here, processing required for monitoring traffic) based on the received packet, timestamp, and flow ID. The processing execution unit 14 includes a counting unit 14A, a packet capture unit 14B, and anomaly detection unit 14C as calculation units that execute the processing.

[0021] The counting unit 14A counts the number of times a flow ID is input from the packet identification unit 13 for each flow ID, i.e., for each flow. The counted number of times of reception can also be said to be the number of packets received by the receiving unit 11, i.e., the number of packets. The counting unit 14A measures the amount of data based on the packets from the receiving unit 11 and accumulates the measured amount of data for each flow. For each fixed period, the counting unit 14A outputs the number of packets and the amount of data counted or accumulated for each flow to the output unit 15 as the accumulation result. The processing result may be either the number of packets or the amount of data.

[0022] The packet capture unit 14B executes packet capture to store the packets from the receiving unit 11 in the storage unit 19. Furthermore, the packet capture unit 14B assigns a time stamp from the time stamp unit 12 to the packets.

[0023] The anomaly detection unit 14C detects a traffic anomaly (such as a microburst) for each flow based on some discriminant on the basis of the packets stored in the storage unit 19, the counting result by the counting unit 14A, the timestamp from the timestamp unit 12, etc. When the anomaly detection unit 14C detects a traffic anomaly, it outputs to the output unit 15 anomaly occurrence information indicating that a traffic anomaly has occurred. The anomaly occurrence information may include information on the type of the traffic anomaly that has occurred.

[0024] The output unit 15 outputs the processing result by the processing execution unit 14, here, the counting result from the counting unit 14A and the abnormality occurrence information from the abnormality detection unit 14C. The output unit 15 displays the counting result and / or the abnormality occurrence information on, for example, a display unit provided in the data processing device 10. For example, the output unit 15 outputs the abnormality occurrence information to a predetermined processing unit provided inside or outside the data processing device 10, and the processing unit may execute a process for cutting off communication for a flow in which a traffic abnormality has been detected based on the abnormality occurrence information. The output unit 15 may acquire packets captured by the packet capture unit 14B and output the acquired packets to a device external to the data processing device 10.

[0025] In this embodiment, the receiving unit 11 is configured so that the processing time of the receiving process (hereinafter also referred to as processing time A) is constant regardless of the data content of the packet input to the receiving unit 11. The time stamp unit 12 is also configured so that the processing time of the time stamp process (hereinafter also referred to as processing time B) is constant regardless of the data content of the packet input to the time stamp unit 12. The packet identification unit 13 is also configured so that the processing time of the packet identification process (hereinafter also referred to as processing time C) is constant regardless of the data content of the packet input to the packet identification unit 13. That is, each of the units 11 to 13 completes the necessary processing for any input data (data to be processed) within a constant processing time. The start timing of each of the above processing times A to C is, for example, the timing when the input of data begins. The end timing of each of the above processing times A to C is, for example, the timing when the output of data (processing result, etc.) begins. The processing times A to C may be the same processing time as each other, or may be different from each other.

[0026] The processing time A being the same for any packet means that the processing is completed in a constant processing time regardless of which processing branch is taken in the processing in the receiving unit 11. For example, as shown in the upper diagram of FIG. 4, if the processing time of the processing A after the branch is three clock cycles and the processing time of the processing B is two clock cycles, a delay circuit for one clock cycle is inserted after the processing B as shown in the lower diagram. This makes the processing time after the branch constant. Alternatively, as shown in FIG. 5, a part of the processing A after the branch is parallelized to reduce the processing time to two clock cycles. This makes the processing time after the branch constant. The same applies to the time stamp unit 12 and the packet identification unit 13. Note that the adjustment of the processing time is not limited to this. Each unit 11 to 13 may measure the time from the start of processing using a timer or the like, and even if the processing is completed, wait for the output of the data of the processing result until the time from the start of processing measured by the timer or the like reaches the processing times A to C.

[0027] As shown in FIG. 6, the receiving unit 11 executes a receiving process at a processing time A. A packet received in the receiving process is input to the time stamping unit 12 and the process execution unit 14 (timing T2). The time stamping unit 12 executes a time stamping process at a processing time B, triggered by the input of a packet from the receiving unit 11. The time stamp added in the time stamping process is input to the process execution unit 14, and the packet to which the time stamp is to be added is input to the packet identification unit 13 (timing T3). The packet identification unit 13 executes a packet identification process at a processing time C, triggered by the input of a packet from the time stamping unit 12. The flow ID and the like acquired in the packet identification process are input to the process execution unit 14 (timing T4).

[0028] The processing execution unit 14 can identify which data has been input to itself in relation to the processing times A to C. This is because the processing times A to C are constant as described above. The receiving unit 11 transmits a packet reception notification to the processing execution unit 14, for example, at the start of the packet reception process (timing T1).

[0029] After the reception notification is transmitted, a packet is always input to the processing execution unit 14 after the processing time A has elapsed (timing T2). Therefore, the processing execution unit 14 recognizes the data input after the processing time A has elapsed after receiving the reception notification as a packet. The processing execution unit 14 performs packet capture, for example, by using the packet capture unit 14B to store the packet in the storage unit 19.

[0030] A timestamp is always input to the processing execution unit 14 after the processing time B has elapsed since the input of the packet (timing T3). Therefore, the processing execution unit 14 recognizes the data input after the processing time B has elapsed since the input of the packet as the timestamp added to the packet. For example, the processing execution unit 14 associates the timestamp input from the timestamp unit 12 with the packet and stores the timestamp in the storage unit 19 so that the timestamp is added to the packet saved in the storage unit 19 by the packet capture unit 14B. Note that the processing execution unit 14 may hold the packet, and add a timestamp to the packet when the timestamp is input, and store the packet in the storage unit 19.

[0031] After the timestamp is input, the flow ID and other information obtained by the packet identification process are always input to the process execution unit 14 after the processing time C has elapsed (timing T4). Therefore, the process execution unit 14 recognizes the data input after the processing time C has elapsed after the timestamp is input as the flow ID of the flow to which the current packet belongs. The process execution unit 14 executes the process of the counting unit 14A and the process of the anomaly detection unit 14C based on the flow ID and other information.

[0032] The processing execution unit 14 can determine which data arrived at which timing based on the processing times A to C of the designed units 11 to 13 for each piece of data input at different timings. Then, the processing execution unit 14 can start processing using at least one of a packet and a timestamp for the packet (for example, packet capture and timestamp assignment in FIG. 6) before a flow ID or the like for a packet is input. This improves the operation efficiency since the processing execution unit 14 does not have to wait to start processing until a flow ID or the like is input as in the above comparative example.

[0033] Furthermore, after receiving a packet from the receiving unit 11, the processing execution unit 14 processes a plurality of pieces of data input at each fixed processing time of the time stamp processing and the packet identification processing as a time stamp for the packet and a flow ID for the packet, thereby preventing the processing execution unit 14 from mistaking the time stamp or flow ID for one packet for the time stamp or flow ID of another packet.

[0034] Furthermore, each of the units 11 to 13 preceding the processing execution unit 14 can sequentially accept subsequent input data, starting from the unit that has completed data output. As shown in Fig. 7, for example, the receiving unit 11 is configured to be able to receive a second packet (see "receiving process" indicated by dotted lines) after outputting a first packet (see "receiving process" indicated by solid lines) before the packet identification process by the packet identification unit 13 is completed. This allows for faster data processing.

[0035] As shown by the dotted line in Fig. 1, at least one of the header analysis unit 13A and the hash calculation unit 13B of the packet identification unit 13 may output data of the processing result to the processing execution unit 14. The processing execution unit 14 may perform a predetermined process based on the input data of the processing result. In such a case, it is preferable that the processing time of each of the units 13A to 13C of the packet identification unit 13 is constant regardless of the content of the data to be processed. The processing execution unit 14 may detect the input of a packet based on the data content, etc., and in this case, which data is the time stamp and flow ID can be identified based on the processing times B and C after the input of the packet.

[0036] 8 shows a hardware configuration diagram of the data processing device 10 configured by a computer. The data processing device 10 includes a processor 101 such as a CPU, a main memory 102 of the processor 101, and a non-volatile storage device 103 that stores programs and various data and configures the storage unit 19 of FIG. 1. The data processing device 10 further includes a NIC (Network Interface Card) 104 that is connected to a communication network NW and relays packets. The processor 101 executes or uses the programs and data stored in the storage device 103 and read into the main memory 102 to operate as the above-mentioned units 11 to 15. The receiving unit 11 and the output unit 15 may be realized by a combination of the processor 101 that executes programs and the NIC 104.

[0037] The data processing device 10 is not limited to the above configuration. The data processing device 10 may include, for example, a receiving unit (the receiving unit 11 in the above configuration) that executes a receiving process to receive and output data (packets in the above configuration), and a first processing unit (the time stamp unit 12 in the above configuration) that executes a first process (time stamp process in the above configuration) on the data output from the receiving unit and outputs a first processing result of the first process (time stamp in the above configuration). The data processing device 10 may include, for example, a second processing unit (the packet identification unit 13 in the above configuration) that executes a second process (packet identification process in the above configuration) on the data after the first process and outputs a second processing result of the second process, and a processing execution unit (the processing execution unit 14 in the above configuration) that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and executes processing based on the received data, the first processing result, and the second processing result (processing by the aggregation unit 14A, the packet capture unit 14B, and the anomaly detection unit 14C in the above configuration). The processing time of at least the first processing and the second processing among the reception processing, the first processing, and the second processing may be constant regardless of the contents of the data to be processed, and the processing execution unit may start processing using at least one of the data and the first processing result for the data (packet capture and time stamp assignment in FIG. 6 in the above) before receiving the second processing result. With such a configuration, the processing execution unit does not wait to start processing until the second processing result is input as in the comparative example, and the operation efficiency of the processing execution unit is improved. Note that specific examples of each of the above processing are not limited to the above embodiment. The data may be image data, the first processing may be a binarization process of an image, and the second processing may be image recognition of the binarized image. The processing execution unit may execute processing to store the image data, the binarized image, and the result of the image recognition in a storage unit.

[0038] The processing execution unit may execute the processing on a plurality of data input at fixed processing times for each of the first processing and the second processing after receiving the data from the receiving unit, as the first processing result and the second processing result for the data, respectively, thereby preventing the first processing result and the second processing result for some data from being confused as the processing result for other data.

[0039] The receiving unit is configured to be able to receive second data as the other data after outputting the first data as the data and before the second process ends, thereby improving the processing speed.

[0040] The present invention is not limited to the above-mentioned embodiment and modification. For example, the present invention includes various modifications to the above-mentioned embodiment and modification that can be understood by a person skilled in the art within the scope of the technical idea of ​​the present invention. The configurations listed in the above-mentioned embodiment and modification can be appropriately combined within a range without contradiction. In addition, it is also possible to delete any of the above-mentioned configurations. The above program may be stored in a non-transitory computer-readable storage medium, not limited to the non-volatile storage device 103. The "device" and "part" may be an entity in which the configuration for realizing the operation is housed in one housing, or an entity (system) in which the configuration for realizing the operation is distributed and housed in multiple housings. [Explanation of symbols]

[0041] 10...data processing device, 11...receiving unit, 12...time stamp unit, 13...packet identification unit, 13A...header analysis unit, 13B...hash calculation unit, 13C...rule matching unit, 14...processing execution unit, 14A...collection unit, 14B...packet capture unit, 14C...anomaly detection unit, 15...output unit, 19...memory unit, 19A...rule table, 101...processor, 102...main memory, 103...memory device, NW...communication network.

Claims

1. A receiving unit that executes a receiving process for receiving and outputting data; a first processing unit that executes a first processing on the data output from the receiving unit and outputs a first processing result of the first processing; a second processing unit that performs a second processing on the data after the first processing and outputs a second processing result of the second processing; a processing execution unit that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and executes processing based on the received data, the first processing result, and the second processing result, a processing time for at least the first processing and the second processing among the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed; the processing execution unit starts processing using at least one of the data and the first processing result for the data before receiving the second processing result. Data processing device.

2. The data is a packet transmitted through a communication network, the first processing unit is a time stamp unit that executes, as the first processing, a time stamp process of generating a time stamp for the packet output from the receiving unit and outputting the generated time stamp as a result of the first processing, the second processing unit is a packet identification unit that executes, as the second processing, a packet identification process of identifying a flow to which the packet for which the timestamp is generated belongs, and outputting flow information specifying the identified flow as a result of the second processing.

2. A data processing apparatus according to claim 1.

3. the processing execution unit executes the processing of a plurality of data input at each fixed processing time of the first processing and the second processing after receiving the data from the receiving unit, as the first processing result and the second processing result for the data, respectively.

3. A data processing device according to claim 1 or 2.

4. The receiving unit is configured to be able to receive second data as the other data before the second process is completed after outputting first data as the data. A data processing device according to any one of claims 1 to 3.

5. The data is a packet transmitted through a communication network, the second processing unit includes a header analysis unit that analyzes a header of the packet to extract a header field value, a hash calculation unit that hashes the extracted header field value, and a rule matching unit that compares the hashed header field value with a predetermined rule to obtain flow information that identifies a flow to which the packet belongs. A data processing device according to any one of claims 1 to 4.

6. a receiving step of executing a receiving process for receiving and outputting data; a first processing step of executing a first processing on the data output in the receiving step and outputting a first processing result of the first processing; a second processing step of performing a second processing on the data after the first processing and outputting a second processing result of the second processing; a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result, a processing time for at least the first processing and the second processing among the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed; In the processing execution step, processing using at least one of the data and the first processing result on the data is started before the second processing result is received. Data processing methods.

7. On the computer, a receiving step of executing a receiving process for receiving and outputting data; a first processing step of executing a first processing on the data output in the receiving step and outputting a first processing result of the first processing; a second processing step of performing a second processing on the data after the first processing and outputting a second processing result of the second processing; a processing execution step of receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and executing processing based on the received data, the first processing result, and the second processing result, a processing time for at least the first processing and the second processing among the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed; In the processing execution step, processing using at least one of the data and the first processing result on the data is started before the second processing result is received. program.

Images