Dynamic processing device and method of data stream, electronic device, medium and product

Abstract

The invention relates to a dynamic processing device for data streams, which comprises an optical bypass device, a converging and shunting single board, a DPI processing single board and an exchange processing module, the optical bypass device is connected with the exchange processing module through an RL1 / RL2 serial interface, the converging and shunting single board is connected with the exchange processing module through an R1 / R2 interface, and the DPI processing single board is connected with the exchange processing module through an RL1 / RL2 serial interface. The DPI processing single board is connected with the switching processing module through a network input port GR1 / GR2, and the optical bypass device, the converging and shunting single board and the DPI processing single board are connected in parallel. The optical bypass device, the converging and shunting single board and the DPI processing single board are integrated on one backboard, and the data streams of the converging and shunting single board and the DPI processing single board are effectively distributed through the exchange processing module, so that external optical fiber connection is not needed, and propagation paths are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of communication equipment, and in particular to a dynamic processing device, method, electronic device, medium and product for data stream. Background Art

[0002] Optical bypass devices and convergence / splitting devices are commonly used in communications networks. DPI servers are typically deployed as customized systems based on general-purpose servers. Information security management systems require these devices, as well as DPI servers. Each device operates independently, requiring external fiber optic connections. This results in long data transmission paths and high latency, and policy configurations require manual synchronization, preventing real-time linkage. Furthermore, their deployment requires significant equipment room space and supporting systems for proper operation. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, electronic device, medium and product for dynamic processing of data streams to solve the problems raised in the above background technology.

[0004] A first aspect of the present invention provides a dynamic processing device for data streams, comprising an optical bypass, a convergence and diversion board, a DPI processing board, and a switching processing module. The optical bypass is connected to the switching processing module via an RL1 / RL2 serial interface, the convergence and diversion board is connected to the switching processing module via an R1 / R2 interface, and the DPI processing board is connected to the switching processing module via a network input port GR1 / GR2. The optical bypass, convergence and diversion board, and DPI processing board are connected in parallel.

[0005] In a possible implementation, a Retimer chip is respectively provided between the optical bypass and the switching processing module, between the convergence and distribution board and the switching processing module, and between the DPI processing board and the switching processing module.

[0006] A second aspect of the present invention provides a method for dynamically processing a data stream, comprising: S1 real-time detection of the working mode of the optical bypass device, determine the optical bypass device is in the bypass state or access state; S2. When in bypass mode, the optical bypass unit's built-in splitter fully mirrors the core network traffic and transmits the mirrored traffic to the aggregation and distribution board via a direct backplane connection. S3. Load the first-level rule matching policy on the aggregation and distribution board, perform protocol filtering, traffic labeling, and load balancing on the mirrored traffic, and output the labeled traffic to the DPI processing board. S4 when in access state, control the optical bypass will forward all traffic to the convergence and diversion board, activate the second-level rule matching strategy, the second-level rule matching strategy contains the dynamic priority decision logic of the port effective rules and switching processing module rules; S5. Real-time monitoring of the DPI processing board's service load via the switching processing module dynamically adjusts the weights and traffic distribution ratios of each rule in the second-level rule matching policy of the convergence and distribution board; In a possible implementation, after step S5, the method further includes: S6. Update the configuration instructions of the backplane control channel based on the adjusted policy parameters to achieve coordinated policy execution among multiple boards.

[0007] In a possible implementation, the mirrored traffic in step S2 undergoes signal reshaping via a Retimer chip in the backplane, and an identification field including a timestamp and a source port number is added during transmission.

[0008] In one possible implementation, the first-level rule matching strategy includes: Pre-classify the mirrored traffic based on the five-tuple feature and generate metadata tags including service type and traffic size; A dynamic hashing algorithm is used to distribute traffic of different service types to the GR1 / GR2 interface of the DPI processing board. The distribution weight is dynamically calculated based on the real-time throughput of the interface.

[0009] In one possible implementation, the dynamic priority determination logic includes: When it is detected that the rule matching success rate of the R1 / R2 port is lower than the preset matching threshold, the rule matching right is automatically switched to the switching processing module; During the switching process, the matching result cache of the port rules is retained and cross-validated with the matching result of the switching processing module.

[0010] In one possible implementation, step S5 includes: The backplane management bus is used to collect the decryption chip utilization, GPU memory occupancy, and message queue depth of the DPI processing board; When any indicator exceeds the dynamic warning threshold, the convergence and distribution board is triggered to enable traffic sampling compression mode. The sampling rate is negatively correlated with the overload degree.

[0011] In a possible implementation, the coordinated execution of the strategies in step S6 includes: When switching from the bypass state to the access state, a link switching instruction is sent to the optical bypass through the backplane control channel, and the traffic buffer queue of the aggregation and distribution board is cleared synchronously; After the state switching is completed, the switching processing module injects simulated test traffic into the DPI processing board to verify the integrity of the processing link.

[0012] According to a third aspect of the present invention, there is provided an electronic device, comprising: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the aforementioned method.

[0013] According to a fourth aspect of the present invention, a computer-readable medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the aforementioned method.

[0014] According to a fifth aspect of the present invention, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By integrating the optical bypass, convergence and distribution board, and DPI processing board on a single backplane, and effectively distributing the data streams of the convergence and distribution board and the DPI processing board through the switching processing module, no external optical fiber connection is required, thus reducing the transmission path; 2. Applying a two-level rule matching strategy can effectively achieve coordinated execution of policies across multiple boards, rationally manage the effective load of each board, and maximize resource utilization on each board. 3. The use of dynamic priority determination logic and real-time load feedback can more reasonably distribute data streams among various boards. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 An exemplary schematic diagram of a data stream dynamic processing device provided for some embodiments of the present application; Figure 2 A schematic diagram of a backplane plug-in method for a data stream dynamic processing device provided in some embodiments of the present application; Figure 3 A flowchart of a method for dynamically processing data streams provided in some embodiments of the present application; Figure 4 is a schematic diagram of an exemplary structure of a processor and memory according to the present application; Figure 5 Schematic diagram of an exemplary structure of an electronic device according to the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, ordinary technicians in this field will be able to understand the present invention in detail without any problems. All other embodiments obtained under the premise of creative work shall fall within the scope of protection of the present invention.

[0018] It should be noted that the serial numbers assigned to the components in the embodiments of the present invention, such as "first" and "second", are only used to distinguish the objects being described and do not have any order or technical meaning.

[0019] The following combination Figure 1 and 2 The structure of the dynamic data stream processing device is described. RX represents downloading, TX represents uploading, and the dotted lines in the figure represent network interface connections.

[0020] The dynamic data flow processing device of the present invention includes an optical bypass, a convergence and diversion board, a DPI processing board, and a switching processing module. The optical bypass is connected to the switching processing module via an RL1 / RL2 serial interface, the convergence and diversion board is connected to the switching processing module via an R1 / R2 interface, and the DPI processing board is connected to the switching processing module via a network input port GR1 / GR2. The optical bypass, convergence and diversion board, and DPI processing board are connected in parallel. A retimer chip is provided between the optical bypass and the switching processing module, between the convergence and diversion board and the switching processing module, and between the DPI processing board and the switching processing module.

[0021] In the present invention, the optical bypass, the convergence and diversion board, and the DPI processing board are connected in parallel, and their respective working states are adjusted and allocated by the switching processing module. The data to be processed (core NE1 and NE2 as shown in the figure) is first transmitted to the optical bypass. The optical bypass can switch between the bypass state and the access state according to the instructions given by the switching processing module. Different paths are used to transmit the data transmitted by core NE1 and NE2 according to different states. When the optical bypass is in bypass mode, all data is directly mirrored and transmitted to the convergence and diversion board. When the optical bypass is in the access state, the switching processing module will reasonably allocate the data volume on the convergence and diversion board and the DPI board based on the load of the DPI board. The retimer chip in the present invention can maintain the stability of the data flow and facilitate real-time processing by the switching processing module. The switching processing module can adopt various commercially available ARM architecture processors.

[0022] In this invention, the optical bypass, convergence and diversion boards, DPI processing board, and switching processing module are all plugged into a single backplane, achieving integration. The backplane has eight service slots. The optical bypass board supports line protection for two links; the convergence and diversion board supports multiple input and output ports; and the DPI processing board supports four 100G interfaces (two external and two internal, interconnected with the switching processing module) and two 10G interfaces. The optical bypass board, convergence and diversion board, and DPI processing board are connected to the backplane via physical interfaces. The backplane is then connected to the switching processing module, and the entire system operates through the backplane interfaces and the switching processing module.

[0023] The present invention also provides a method for processing dynamic data streams using the aforementioned dynamic data stream processing device. Figure 3 The method is described in detail: A dynamic data stream processing method, comprising: S1 real-time detection of the working mode of the optical bypass device, determine the optical bypass device is in the bypass state or access state; The working status of the optical bypass can be determined by checking the indicator light on the optical bypass or performing data flow detection on the optical bypass. If bidirectional data flow is generated in the optical bypass, it is in the access state; otherwise, it is in the bypass state.

[0024] S2. When in bypass mode, the optical bypass unit's built-in splitter fully mirrors core network traffic and transmits the mirrored traffic to the aggregation and distribution board via the backplane direct connection. Physical isolation of bypass mirrored traffic and access direct traffic effectively prevents the spread of single-point failures.

[0025] The mirrored traffic is reshaped by the retimer chip in the backplane, and an identification field containing a timestamp and source port number is added during transmission. Adding the timestamp and source port number identification fields facilitates effective identification of the data source during subsequent data transmission and processing. The retimer chip in this application supports the PCIe 4.0 protocol and is suitable for signal reshaping interfaces at speeds of 100Gbps and above.

[0026] S3. Load the first-level rule matching policy on the aggregation and distribution board, perform protocol filtering, traffic labeling, and load balancing on the mirrored traffic, and output the labeled traffic to the DPI processing board. The first-level rule matching strategy includes: Mirrored traffic is pre-classified based on its five-tuple characteristics, generating metadata tags that include service type and traffic size. The five-tuple is a set of five parameters: source IP address, source port, destination IP address, destination port, and transport layer protocol. Service types can be categorized by priority: urgent, priority, and normal. During data flow transmission, urgent data is transmitted first, followed by priority data, and finally normal data.

[0027] A dynamic hashing algorithm is used to distribute traffic of different service types to the GR1 / GR2 interface of the DPI processing board. The distribution weight is dynamically calculated based on the real-time throughput of the interface.

[0028] Specifically, the following features are extracted from each traffic data packet to generate a composite hash key K. The hash value is generated by CRC32 checksum, and then the weight dynamic mapping algorithm is used. First, the total weight is calculated: W 总 =W1+W2, W1 is the weight of GR1 interface, W2 is the weight of GR2 interface; then calculate the interface intervals to which GR1 interface and GR2 interface belong respectively, and the interface interval of GR1 interface is [0, W1 / W 总 *2 64 ), the GR2 interface interval is (W2 / W 总 *2 64 , 2 64 ), perform 2 on the hash value K 64 Operation, rotate the output interface according to the operation result.

[0029] S4 when in access state, control the optical bypass will forward all traffic to the convergence and diversion board, activate the second-level rule matching strategy, the second-level rule matching strategy contains the dynamic priority decision logic of the port effective rules and switching processing module rules; The second-level rule matching strategy is used to match R1 / R2 ports. When matching R1 / R2 ports, the matching can be performed in the same manner as the GR1 / GR2 interface described above. Furthermore, the strategy can also include dynamic priority determination logic for port validation rules and switching processing module rules, wherein the dynamic priority determination logic includes: When it is detected that the R1 / R2 port rule matching success rate is lower than the preset matching threshold, the rule matching right is automatically switched to the switching processing module; the switching processing module effectively allocates the use of the port, which can effectively improve the efficiency of the allocation of the use of the aggregation and diversion board ports and reduce the workload of the main processor.

[0030] During the switchover process, the port rule matching results are cached and cross-validated with the matching results from the switch processing module. Cross-validation involves extracting matching results for the same traffic from both the port cache and the switch cache based on the five-tuple hash value and a timestamp window (±1ms). The rule IDs and action instructions in the two caches are then compared for consistency.

[0031] S5. Real-time monitoring of the DPI processing board's service load via the switching processing module dynamically adjusts the weights and traffic distribution ratios of each rule in the second-level rule matching policy of the convergence and distribution board; Specifically, the following methods can be used: the decryption chip utilization, GPU memory occupancy and message queue depth of the DPI processing board are collected through the backplane management bus; When any metric exceeds the dynamic warning threshold, the convergence and distribution boards are triggered to enable traffic sampling compression mode. The sampling rate is negatively correlated with the degree of overload. In overload scenarios, dynamic traffic sampling compression is enabled, improving measured resource utilization and ensuring business continuity under high throughput conditions.

[0032] S6. Update the configuration instructions of the backplane control channel based on the adjusted policy parameters to achieve coordinated policy execution among multiple boards.

[0033] When switching from the bypass state to the access state, a link switching instruction is sent to the optical bypass through the backplane control channel, and the traffic buffer queue of the aggregation and distribution board is cleared synchronously; After the state switching is completed, the switching processing module injects simulated test traffic into the DPI processing board to verify the integrity of the processing link.

[0034] In addition, some embodiments of the present application further provide an electronic device. The electronic device may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device may also be various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0035] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein the computer program instructions, when executed, enable the processor to perform the steps of the method provided in any one or more of the above embodiments. Figure 4 An exemplary structural diagram of the electronic device is disclosed. Figure 4 As shown, the electronic device includes: One or more processors 11, memory 12, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0036] The electronic device may further include: an input device 13 and an output device 14. The processor 11, the memory 12, the input device 13 and the output device 14 may be connected via a bus or other means. Figure 5 The bus connection is taken as an example.

[0037] The input device 13 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, and the like. The output device 14 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0038] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium. When executed by a processor, the computer program / instruction implements the steps of the method provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments, or it may exist independently and not be incorporated into the device. The computer-readable medium carries one or more computer-readable instructions.

[0039] The memory 12 can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor 11 executes the non-transitory software programs, instructions, and modules stored in the memory 12 to execute various functional applications and data processing of the server, thereby implementing the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0040] The computer program product provided by the embodiment of the present application includes one or more computer programs / instructions, and when the computer program / instructions are executed by the processor, all or part of the process or function described in the embodiment of the present application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium. For example, the computer instruction can be transmitted from a website, a computer, a server or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, a computer, a server or a data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive so lid state disk (SSD)) etc.

[0041] It is obvious to those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

Claims

1. A data stream dynamic processing device, characterized in that: It includes an optical bypass, a convergence and diversion board, a DPI processing board and a switching processing module. The optical bypass is connected to the switching processing module through the RL1 / RL2 serial interface, the convergence and diversion board is connected to the switching processing module through the R1 / R2 interface, and the DPI processing board is connected to the switching processing module through the network input port GR1 / GR2. The optical bypass, convergence and diversion board and DPI processing board are connected in parallel.

2. The device according to claim 1, characterized in that A Retimer chip is respectively provided between the optical bypass and the switching processing module, between the convergence and distribution board and the switching processing module, and between the DPI processing board and the switching processing module.

3. A method for dynamic processing of data streams, characterized in that: Applied to the device of claim 1 or 2, the method comprises: S1 real-time detection of the working mode of the optical bypass device, determine the optical bypass device is in the bypass state or access state; S2. When in bypass mode, the optical bypass unit's built-in splitter fully mirrors the core network traffic and transmits the mirrored traffic to the aggregation and distribution board via a direct backplane connection. S3. Load the first-level rule matching policy on the aggregation and distribution board, perform protocol filtering, traffic labeling, and load balancing on the mirrored traffic, and output the labeled traffic to the DPI processing board. S4 when in access state, control the optical bypass will forward all traffic to the convergence and diversion board, activate the second-level rule matching strategy, the second-level rule matching strategy contains the dynamic priority decision logic of the port effective rules and switching processing module rules; S5. The switching processing module monitors the traffic load of the DPI processing board in real time and dynamically adjusts the weights and traffic distribution ratios of each rule in the second-level rule matching policy of the aggregation and distribution board.

4. The method according to claim 3, characterized in that The mirrored traffic in step S2 undergoes signal reshaping via a Retimer chip in the backplane, and an identification field including a timestamp and a source port number is added during transmission.

5. The method according to claim 3, characterized in that The first-level rule matching strategy includes: Pre-classify the mirrored traffic based on the five-tuple feature and generate metadata tags including service type and traffic size; A dynamic hashing algorithm is used to distribute traffic of different service types to the GR1 / GR2 interface of the DPI processing board. The distribution weight is dynamically calculated based on the real-time throughput of the interface.

6. The method according to claim 3, characterized in that The dynamic priority determination logic includes: When it is detected that the rule matching success rate of the R1 / R2 port is lower than the preset matching threshold, the rule matching right is automatically switched to the switching processing module; During the switching process, the matching result cache of the port rules is retained and cross-validated with the matching result of the switching processing module.

7. The method according to claim 3, wherein: The step S5 comprises: The backplane management bus is used to collect the decryption chip utilization, GPU memory occupancy, and message queue depth of the DPI processing board; When any indicator exceeds the dynamic warning threshold, the convergence and distribution board is triggered to enable traffic sampling compression mode. The sampling rate is negatively correlated with the overload degree.

8. An electronic device, characterized in that: The electronic device comprises: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the method according to any one of claims 3 to 7.

9. A computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 3 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 3 to 7 are implemented.

Images