Data stream transmission device and method based on optical bypass, electronic device and medium

Abstract

The invention relates to a data stream transmission device based on an optical bypass, which comprises an optical bypass device, a converging and shunting single board and a DPI processing single board, an L1 interface of the optical bypass device is connected with an I1 input port of the converging and shunting single board through an optical fiber, an L2 interface of the optical bypass device is interconnected with an I2 input port of the converging and shunting single board through an optical fiber, and the DPI processing single board is connected with the converging and shunting single board through an optical fiber. The G1 interface of the DPI processing single board is interconnected with the O1 interface of the convergence and shunting single board through an optical fiber, and the G2 interface of the DPI processing single board is interconnected with the O2 interface of the convergence and shunting single board through an optical fiber. The optical bypass device, the converging and shunting single board and the DPI processing single board are integrated on one backboard and are connected in series through the optical fibers, external optical fiber connection is not needed, propagation paths are reduced, and the load and the data flow of each single board can be conveniently adjusted in time.

Description

Technical Field

[0001] The present invention relates to the technical field of communication equipment, and in particular to data stream transmission equipment, methods, electronic equipment and media based on optical bypass. Background Art

[0002] Optical bypass devices and convergence / splitting devices are commonly used in communications networks. DPI servers are typically deployed as customized systems based on general-purpose servers. Information security management systems require these devices, as well as DPI servers. Each device operates independently, requiring external fiber optic connections. This results in long data transmission paths and high latency, and policy configurations require manual synchronization, preventing real-time linkage. Furthermore, their deployment requires significant equipment room space and supporting systems for proper operation. Summary of the Invention

[0003] The object of the present invention is to provide a data stream transmission device, method, system, device and storage medium based on optical bypass to solve the problems raised in the above background technology.

[0004] A first aspect of the present invention provides a data stream transmission device based on optical bypass, comprising an optical bypass, a convergence and diversion board, and a DPI processing board, wherein the L1 interface of the optical bypass is connected to the I1 input port of the convergence and diversion board via optical fiber, the L2 interface of the optical bypass is interconnected with the I2 input port of the convergence and diversion board via optical fiber, the G1 interface of the DPI processing board is interconnected with the O1 interface of the convergence and diversion board via optical fiber, and the G2 interface of the DPI processing board is interconnected with the O2 interface of the convergence and diversion board via optical fiber.

[0005] In a possible implementation manner, a T1 / T2 interface is provided on the DPI processing board, and the DPI processing board is connected to the shared layer platform via the T1 / T2 interface.

[0006] A second aspect of the present invention provides a data stream transmission method based on optical bypass, comprising: S1 real-time detection of the working mode of the optical bypass device, determine the optical bypass device is in the bypass state or access state; S2. When in bypass mode, the optical bypass unit's built-in splitter fully mirrors the core network traffic and transmits the mirrored traffic to the aggregation and distribution board via a direct backplane connection. S3. Load the first-level rule matching policy on the aggregation and distribution board, perform protocol filtering, traffic labeling, and load balancing on the mirrored traffic, and output the labeled traffic to the DPI processing board. S4 when in access state, control the optical bypass to forward all traffic to the convergence and diversion board, activate the second-level rule matching strategy, the second-level rule matching strategy; S5. Real-time monitoring of the DPI processing board's service load, dynamically adjusting the data flow distribution ratio in the aggregation and distribution board, DPI processing board, and optical bypass; S6. Update the configuration instructions of the backplane control channel according to the allocation ratio to achieve coordinated execution of policies among multiple boards.

[0007] In one possible implementation, transmitting the mirrored traffic to the convergence and distribution board via the backplane direct connection channel includes: During the transmission process, an identification field including a timestamp and a source port number is added, and the identification field is embedded in an extended header of an Ethernet frame in an in-band communication manner.

[0008] In one possible implementation, the first-level rule matching strategy includes: Pre-classify the mirrored traffic based on the five-tuple feature and generate metadata tags including service type and traffic size; A dynamic hashing algorithm is used to distribute traffic of different service types to the G1 / G2 interface of the DPI processing board. The distribution weight is dynamically calculated based on the real-time throughput of the interface.

[0009] In one possible implementation, the second-level rule matching strategy includes: When the data flow enters the access state, the port local rule or global rule is automatically selected to take effect according to the preset rule priority judgment logic; During the rule switching process, the matching result cache of the port rule is retained and cross-verified with the global rule matching result, and redirection processing is initiated for conflicting data flows.

[0010] In one possible implementation, step S5 includes: The backplane management bus is used to collect the decryption chip utilization, GPU memory occupancy, and message queue depth of the DPI processing board; When any indicator exceeds the dynamic warning threshold, the convergence and distribution board is triggered to enable traffic sampling compression mode. The sampling rate is negatively correlated with the overload degree.

[0011] In a possible implementation, the coordinated execution of the strategies in step S6 includes: When switching from the bypass state to the access state, a link switching instruction is sent to the optical bypass through the backplane control channel, and the traffic buffer queue of the aggregation and distribution board is cleared synchronously; After the state switch is completed, simulated test traffic is injected into the DPI processing board to verify the integrity of the processing link.

[0012] A third aspect of the present invention provides an electronic device comprising: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the aforementioned method.

[0013] According to a fourth aspect of the present invention, a computer-readable medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the aforementioned method.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. By integrating the optical bypass, convergence and distribution board, and DPI processing board on a single backplane and connecting them in series using optical fiber, no external optical fiber connection is required, the transmission path is reduced, and the load and data flow of each board can be adjusted in a timely manner; 2. Applying a two-level rule matching strategy can effectively achieve coordinated execution of policies across multiple boards, rationally manage the effective load of each board, and maximize resource utilization on each board. 3. By analyzing the real-time load and dynamically adjusting the data flow distribution strategy, the data flow can be distributed more reasonably among the various boards. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 An exemplary schematic diagram of a data stream transmission device based on optical bypass is provided for some embodiments of the present application; Figure 2 A schematic diagram of a backplane plug-in method for a data stream transmission device based on optical bypass provided in some embodiments of the present application; Figure 3 A flowchart of a data stream transmission method based on optical bypass is provided for some embodiments of the present application; Figure 4 is a schematic diagram of an exemplary structure of a processor and memory according to the present application; Figure 5 Schematic diagram of an exemplary structure of an electronic device according to the present application. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0017] It should be noted that the serial numbers assigned to the components in the embodiments of the present invention, such as "first" and "second", are only used to distinguish the objects being described and do not have any order or technical meaning.

[0018] The following combination Figure 1 and 2 The structure of a data stream transmission device based on optical bypass is described. RX represents download, TX represents upload, and the dotted lines in the figure represent network interface connections.

[0019] The optical bypass-based data stream transmission device of the present invention includes an optical bypass, a convergence and diversion board, and a DPI processing board. The L1 interface of the optical bypass is connected to the I1 input port of the convergence and diversion board via an optical fiber. The L2 interface of the optical bypass is interconnected with the I2 input port of the convergence and diversion board via an optical fiber. The G1 interface of the DPI processing board is interconnected with the O1 interface of the convergence and diversion board via an optical fiber. The G2 interface of the DPI processing board is interconnected with the O2 interface of the convergence and diversion board via an optical fiber. The DPI processing board is provided with a T1 / T2 interface, and the DPI processing board is connected to a shared layer platform via the T1 / T2 interface.

[0020] In the present invention, an optical bypass, a convergence and diversion board, and a DPI processing board are connected in series via optical fiber. The transmission efficiency of data flows within the optical bypass, convergence and diversion board, and DPI processing board can be improved by adjusting and allocating the operating status of each board. Data requiring processing (as shown in the figure, core NE1 and NE2) is first transmitted to the optical bypass. The optical bypass can switch between bypass and access states based on instructions from the switching processing module. Data transmitted by core NE1 and NE2 is transmitted using different paths depending on the state. When the optical bypass is in bypass mode, all data is directly mirrored and transmitted to the convergence and diversion board. When the optical bypass is in access mode, the switching processing module rationally allocates the data flow between the convergence and diversion board and the DPI processing board based on the DPI board's load and other factors. When the optical bypass is in the access state, by formulating effective strategies for the convergence and diversion board, the data flow can be reasonably diverted among the optical bypass, convergence and diversion board, and DPI processing board, so that the three will not be overloaded, thereby improving the transmission efficiency of the data flow.

[0021] In this invention, the optical bypass, convergence and diversion board, and DPI processing board are all plugged into a single backplane, achieving integration. The backplane has eight service slots. The optical bypass board supports line protection for two links; the convergence and diversion board supports multiple input and output ports; and the DPI processing board supports two 100G interfaces. The optical bypass board, convergence and diversion board, and DPI processing board are connected to the backplane via physical interfaces. The backplane interface also connects to the logic control unit, which controls the entire system.

[0022] The present invention also provides a method for transmitting data stream based on optical bypass by using the aforementioned data stream transmission device based on optical bypass. Figure 3 The method is described in detail: A data stream transmission method based on optical bypass, comprising: S1 real-time detection of the working mode of the optical bypass device, determine the optical bypass device is in the bypass state or access state; The working status of the optical bypass can be determined by checking the indicator light on the optical bypass or performing data flow detection on the optical bypass. If bidirectional data flow is generated in the optical bypass, it is in the access state; otherwise, it is in the bypass state.

[0023] S2. When in bypass mode, the optical bypass unit's built-in splitter fully mirrors core network traffic and transmits the mirrored traffic to the aggregation and distribution board via a direct backplane connection. Physical isolation of bypass mirrored traffic from access direct traffic effectively prevents the spread of single-point failures. The built-in splitter allows the splitting ratio between the primary and mirrored links to be determined based on actual needs.

[0024] During transmission, an identification field containing a timestamp and source port number is added. This identification field is embedded in the extended header of the Ethernet frame via in-band communication. By extending the header directly within the existing Ethernet frame, communication parameters or management information can be transmitted without establishing a separate control channel. This approach avoids the introduction of additional communication protocol layers and reduces the complexity of the network architecture.

[0025] S3. Load the first-level rule matching policy on the aggregation and distribution board, perform protocol filtering, traffic labeling, and load balancing on the mirrored traffic, and output the labeled traffic to the DPI processing board. Mirrored traffic is pre-classified based on its five-tuple characteristics, generating metadata tags containing service type and traffic size. The five-tuple is a set of five parameters: source IP address, source port, destination IP address, destination port, and transport layer protocol. Service types can be categorized by priority: emergency, priority, and standard. During optical bypass-based data flow transmission, emergency data is transmitted first, followed by priority data, and finally standard data.

[0026] A dynamic hashing algorithm is used to distribute traffic of different service types to the G1 / G2 interface of the DPI processing board. The distribution weight is dynamically calculated based on the real-time throughput of the interface.

[0027] Specifically, the following features are extracted from each traffic data packet to generate a composite hash key K. The hash value is generated by CRC32 checksum, and then the weight dynamic mapping algorithm is used. First, the total weight is calculated: W 总 =W1+W2, W1 is the weight of G1 interface, W2 is the weight of G2 interface; then calculate the interface intervals to which G1 interface and G2 interface belong respectively, and the interface interval of G1 is [0, W1 / W 总 *2 64 ), G2 interface interval is (W2 / W 总 *2 64 , 2 64 ), perform 2 on the hash value K 64 Operation, rotate the output interface according to the operation result.

[0028] S4 when in access state, control the optical bypass to forward all traffic to the convergence and diversion board, activate the second-level rule matching strategy, the second-level rule matching strategy; The second-level rule matching strategy can be implemented as follows: When the data flow enters the access state, the port local rule or global rule is automatically selected to take effect according to the preset rule priority judgment logic; During the rule switching process, the matching result cache of the port rule is retained and cross-verified with the global rule matching result, and redirection processing is initiated for conflicting data flows.

[0029] The following further details the process of the second season rule matching strategy: In the access state initialization phase, a mapping relationship table between the port local rule base and the global rule base is constructed. The global rule base is stored in the non-volatile memory of the convergence and distribution board; Create a rule matching context for each data flow, including the five-tuple hash value, the first match timestamp, and the rule hit counter; When data streams arrive, the following operations are performed in parallel: Port local rule matching: The multi-core processor of the aggregation and distribution board matches the port local rules based on the five-tuple characteristics and generates the first matching result - the allow / deny / redirect action and the rule ID; Global rule matching: Submit a query request to the global rule base through the backplane control channel to obtain the second matching result - action and rule ID; The first matching result and the second matching result are written into a dual-port buffer area, wherein the buffer area adopts a ping-pong buffer structure, and each entry contains: Five-tuple hash, first match result, second match result, difference flag, and timestamp The hardware comparator performs real-time comparison of the first and second matching results in the cache entry. The difference determination logic includes: Action consistency check: If the action instructions of the two results are inconsistent, it is marked as a high-risk difference; Rule priority arbitration: When the actions are the same but the rule IDs are different, the rule priority is determined based on the mapping table; Redirect high-risk differential data flows: The disputed traffic is copied to the backup processing link, and the independent detection module performs a third rule match. A majority voting mechanism is used. If two of the three parties agree, the result is adopted and the port's local rule base is updated synchronously. Implement a default security policy for traffic where consensus cannot be reached.

[0030] Collaborative verification of dual rule bases: Through real-time cross-validation of port local rules and global rules, the rule misjudgment rate is effectively reduced. The majority voting fault tolerance mechanism is adopted. Even if one side's rule base is abnormal, the correctness of the decision can still be guaranteed through three-party verification, and the system fault tolerance rate is significantly improved.

[0031] S5. Real-time monitoring of the DPI processing board's service load, dynamically adjusting the data flow distribution ratio in the aggregation and distribution board, DPI processing board, and optical bypass; The backplane management bus is used to collect the decryption chip utilization, GPU memory occupancy, and message queue depth of the DPI processing board; When any metric exceeds the dynamic warning threshold, the convergence and distribution boards are triggered to enable traffic sampling compression mode. The sampling rate is negatively correlated with the degree of overload. In overload scenarios, dynamic traffic sampling compression is enabled, improving measured resource utilization and ensuring business continuity under high throughput conditions.

[0032] S6. Update the configuration instructions of the backplane control channel according to the allocation ratio to achieve coordinated execution of policies among multiple boards.

[0033] Specifically, a hybrid weight algorithm can be used to calculate and dynamically adjust the distribution ratio of data flows between each board. The calculation formula is as follows: is the allocation ratio of the single board, is the maximum throughput of the board, is the actual throughput of the board, is the CPU usage, is the actual bit error rate of the backplane channel, is the backplane channel error threshold, and a, b, and c are constants.

[0034] In addition, some embodiments of the present application further provide an electronic device. The electronic device may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device may also be various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0035] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor performs the steps of the method provided in any one or more of the above embodiments. Figure 4 An exemplary structural diagram of the electronic device is disclosed. Figure 4 As shown, the electronic device includes: One or more processors 11, memory 12, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Among them, the components shown in this article, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0036] The electronic device may further include: an input device 13 and an output device 14. The processor 11, the memory 12, the input device 13 and the output device 14 may be connected via a bus or other means. Figure 5The bus connection is taken as an example.

[0037] The input device 13 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, and the like. The output device 14 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0038] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium. When executed by a processor, the computer program / instruction implements the steps of the method provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments, or it may exist independently and not be incorporated into the device. The computer-readable medium carries one or more computer-readable instructions.

[0039] The memory 12 can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor 11 executes the non-transitory software programs, instructions, and modules stored in the memory 12 to execute various functional applications and data processing of the server, thereby implementing the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0040] It is obvious to those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

Claims

1. A data stream transmission device based on optical bypass, characterized in that: It includes an optical bypass, a convergence and diversion board and a DPI processing board. The L1 interface of the optical bypass is connected to the I1 input port of the convergence and diversion board through optical fiber, the L2 interface of the optical bypass is interconnected with the I2 input port of the convergence and diversion board through optical fiber, the G1 interface of the DPI processing board is interconnected with the O1 interface of the convergence and diversion board through optical fiber, and the G2 interface of the DPI processing board is interconnected with the O2 interface of the convergence and diversion board through optical fiber.

2. The device according to claim 1, characterized in that The DPI processing board is provided with a T1 / T2 interface, and the DPI processing board is connected to the shared layer platform via the T1 / T2 interface.

3. A data stream transmission method based on optical bypass, characterized in that: Applied to the device of claim 1 or 2, the method comprises: S1 real-time detection of the working mode of the optical bypass device, determine the optical bypass device is in the bypass state or access state; S2. When in bypass mode, the optical bypass unit's built-in splitter fully mirrors the core network traffic and transmits the mirrored traffic to the aggregation and distribution board via a direct backplane connection. S3. Load the first-level rule matching policy on the aggregation and distribution board, perform protocol filtering, traffic labeling, and load balancing on the mirrored traffic, and output the labeled traffic to the DPI processing board. S4 when in access state, control the optical bypass to forward all traffic to the convergence and diversion board, activate the second-level rule matching strategy, the second-level rule matching strategy; S5. Real-time monitoring of the DPI processing board's service load, dynamically adjusting the data flow distribution ratio in the aggregation and distribution board, DPI processing board, and optical bypass; S6. Update the configuration instructions of the backplane control channel according to the allocation ratio to achieve coordinated execution of policies among multiple boards.

4. The method according to claim 3, characterized in that The method of transmitting the mirrored traffic to the convergence and distribution board via the backplane direct connection channel includes: During the transmission process, an identification field including a timestamp and a source port number is added, and the identification field is embedded in an extended header of an Ethernet frame in an in-band communication manner.

5. The method according to claim 3, characterized in that The first-level rule matching strategy includes: Pre-classify the mirrored traffic based on the five-tuple feature and generate metadata tags including service type and traffic size; A dynamic hashing algorithm is used to distribute traffic of different service types to the G1 / G2 interface of the DPI processing board. The distribution weight is dynamically calculated based on the real-time throughput of the interface.

6. The method according to claim 3, characterized in that The second-level rule matching strategy includes: When the data flow enters the access state, the port local rule or global rule is automatically selected to take effect according to the preset rule priority judgment logic; During the rule switching process, the matching result cache of the port rule is retained and cross-verified with the global rule matching result, and redirection processing is initiated for conflicting data flows.

7. The method according to claim 3, characterized in that The step S5 comprises: The backplane management bus is used to collect the decryption chip utilization, GPU memory occupancy, and message queue depth of the DPI processing board; When any indicator exceeds the dynamic warning threshold, the convergence and distribution board is triggered to enable traffic sampling compression mode. The sampling rate is negatively correlated with the overload degree.

8. An electronic device, characterized in that: The device comprises: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the method according to any one of claims 3 to 7.

9. A computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 3 to 7.

Images