Method, unit and apparatus for anomaly simulation of large-scale signal processing algorithm service chains

Abstract

This invention relates to a method, unit, and apparatus for anomaly simulation in a large-scale signal processing algorithm business chain, comprising the following steps: Step 1: Based on the testing requirements of the signal processing business chain, identify the signal processing algorithm software that needs to be embedded with the anomaly simulation unit and the business chain links that need to run the anomaly simulation unit independently; Step 2: Configure the anomaly simulation unit, enable the corresponding sub-units, and initialize the behavior mode, utilization rate, and quantity parameters of each sub-unit. This invention supports the simulation of anomalies in various hardware resources such as CPU, GPU, memory, video memory, and network, and can quantitatively assess the propagation impact of anomalies in the business chain.

Description

Technical Field

[0001] This invention belongs to the field of signal processing technology, specifically relating to an anomaly simulation method, unit, and device for a large-scale signal processing algorithm business chain. Background Technology

[0002] As the scale and complexity of signal processing algorithm business chains increase, traditional testing methods are no longer sufficient to meet the assessment needs of module-level anomalies. Existing technologies primarily focus on traffic stress testing and functional testing for business chains, such as the business link load stress testing method proposed in Chinese patent application number 201710114959.1, and the testing method based on massive data proposed in Chinese patent application number 202110970816.7. While these methods can simulate business traffic and data processing capabilities, they cannot simulate abnormal CPU, GPU, memory, and video memory usage of signal processing algorithm modules during runtime, nor can they quantitatively assess the propagation impact of anomalies in one module on other modules.

[0003] In signal processing workflows with extremely high real-time requirements, modules are highly sensitive to hardware resource consumption; resource anomalies can lead to overall performance degradation or even system crashes. Therefore, a testing method is urgently needed that can simulate module-level hardware resource anomalies and assess the impact of anomaly propagation. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, unit and device for anomaly simulation of a large-scale signal processing algorithm business chain, which supports the simulation of anomalies of various hardware resources such as CPU, GPU, memory, video memory and network, and quantitatively assesses the propagation impact of anomalies in the business chain.

[0005] The technical solution of this invention is to provide an anomaly simulation method for a large-scale signal processing algorithm business chain, comprising the following steps:

[0006] Step 1: Based on the signal processing business chain testing requirements, identify the signal processing algorithm software that needs to be embedded with the anomaly simulation unit and the business chain links that need to run the anomaly simulation unit independently;

[0007] Step 2: Configure the anomaly simulation unit, enable the corresponding sub-units, and initialize the behavior mode, utilization rate, and quantity parameters of each sub-unit;

[0008] Step 3: Integrate the anomaly simulation unit and the signal processing algorithm software that needs to be embedded for testing into a container, and configure the corresponding running commands in the container; orchestrate the business chain for the anomaly simulation units that need to run independently; after completion, register the information of the anomaly simulation unit to the business chain management center;

[0009] Step 4: Based on the registration information, construct a directed graph for business chain anomaly testing, consisting of anomaly simulation units in the business chain. The vertices of the graph represent containers composed of a signal processing algorithm software module and anomaly simulation units, or independently running anomaly simulation units. The vertices contain information such as the sub-units enabled by each anomaly simulation unit, the configuration of each sub-unit, and the state of the algorithm software. The edges of the graph represent the impact of a state change of one vertex on the state of another vertex, that is, the impact of module-level anomalies caused by anomaly simulation units on each algorithm software module in the business chain. The initial state of the edges is unknown and is the result of the business chain testing process.

[0010] Step 5: Based on the established business chain anomaly test directed graph, generate multiple sets of anomaly states for each heterogeneous simulation unit. Each set of anomaly states includes different CPU core usage, CPU behavior patterns, CPU utilization, GPU resource usage, GPU behavior patterns, GPU utilization, video memory / memory behavior patterns, usage, video memory / memory bandwidth utilization, network behavior patterns, number of network ports occupied, and bandwidth utilization.

[0011] Step 6: Deploy the business chain to the actual hardware cluster, inject each group of abnormal states in sequence to start the test, record the vertex state changes caused by each group of abnormal states, and reset the entire business chain after each group of abnormal states is tested, and then proceed to the next group of abnormal states test.

[0012] Step 7: After completing the testing of all abnormal states, estimate the edge states based on the recorded vertex state change data to obtain the extent to which each algorithm module affects other software modules when a module-level abnormality occurs.

[0013] This invention can simulate module-level anomalies in the signal processing business chain, supporting the simulation of common anomalies such as CPU / GPU usage, memory usage, and data traffic size, ensuring the completeness of the business chain integration testing phase. This invention supports both embedding conventional algorithm software modules and independent operation within the business chain. When embedding conventional algorithm software, containerized embedding is used, eliminating the need for developers to rewrite or compile the algorithm software code.

[0014] As a preferred embodiment, the method for constructing the directed graph for business chain anomaly testing in step 4 is represented as follows:

[0015]

[0016] Among them, the vertex set , This refers to a container consisting of a signal processing algorithm software module and an anomaly simulation unit, or an independently operating anomaly simulation unit. It represents the number of vertices.

[0017] edge set , Each element represents a vertex. State of Vertex The impact of CPU, GPU, memory, video memory, network, and software status.

[0018] As a preferred option, the state of each vertex is used It means that among them , , , , and These represent the status of the CPU resource usage subunit, GPU resource usage subunit, memory usage subunit, video memory usage subunit, traffic simulation subunit, and the embedded signal processing algorithm software module, respectively.

[0019] As a preferred embodiment, the edge state estimation method in step 7 is as follows:

[0020] The first directed graph test process The vertex state change model for this test is as follows: ,in It represents the real-time state of all vertices. This is the initial state of all vertices. It is a matrix composed of edge states. Indicates the first There were [number] tests, with a total of [number] tests. Next, by solving the formula Calculate the estimated edge states This allows us to determine the extent to which each algorithm module affects other software modules when a module-level anomaly occurs.

[0021] The present invention also provides an anomaly simulation unit for a large-scale signal processing algorithm service chain, comprising: a CPU resource occupancy subunit for simulating the number of CPU cores, behavior patterns, and utilization of the algorithm module; a GPU resource occupancy subunit for simulating the number of GPUs used, behavior patterns, and utilization of the algorithm module; a memory occupancy subunit for simulating the memory behavior patterns, occupancy, and memory bandwidth utilization of the algorithm module; a video memory occupancy subunit for simulating the video memory behavior patterns, video memory occupancy, and video memory bandwidth utilization of the algorithm module; and a traffic simulation subunit for simulating the network behavior patterns, number of network ports occupied, and bandwidth utilization of the algorithm module.

[0022] Preferably, each subunit supports batch initial configuration and remote dynamic configuration, which can be enabled as needed according to testing requirements. That is, before the business chain anomaly simulation unit is enabled, the functions of each of its subunits can be initially configured in batches, and during runtime, batch remote dynamic configuration can be performed via independent network addresses to dynamically simulate specific abnormal situations.

[0023] The present invention also provides an anomaly simulation unit device, including multiple physical link communication interfaces, supporting SRIO, gigabit Ethernet, 10-gigabit Ethernet, 25G optical port and 100G optical port; the device is used to carry the above-mentioned anomaly simulation unit and supports its independent operation in the service chain.

[0024] Furthermore, the device can simulate various types of business chain data flows and work in conjunction with an anomaly simulation unit embedded with conventional algorithm software to complete the test.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] Addressing the shortcomings of existing technologies that focus on traffic stress testing and business processing capacity testing, and are unable to simulate the usage of various hardware resources or assess the impact of module-level anomalies on other modules, this invention addresses the issue that the signal processing business chain modules have extremely high real-time requirements and are highly sensitive to the utilization of hardware resources such as CPU, GPU, memory, video memory, and network. It not only simulates module-level anomalies in the signal processing business chain but also assesses the impact of module-level anomalies on other software modules. Furthermore, the anomaly simulation unit supports both embedding conventional algorithm modules and independent operation based on the unit device. When embedding conventional algorithm software, containerized embedding is used, eliminating the need for developers to rewrite or compile the algorithm software code, making it convenient and highly compatible. Attached Figure Description

[0027] Figure 1 This describes the business link structure in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram illustrating the mutual impact of service links when the CPU experiences a severe abnormality, according to an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram illustrating the mutual impact of service links when the GPU experiences a severe anomaly, according to an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram illustrating the mutual impact of service links when the CPU memory is severely abnormal, according to an embodiment of the present invention.

[0031] Figure 5 This is a schematic diagram illustrating the mutual influence of the service links when the GPU memory is severely abnormal, according to an embodiment of the present invention.

[0032] Figure 6 This is a schematic diagram illustrating the mutual influence of service links when traffic is severely abnormal according to an embodiment of the present invention.

[0033] Figure 7This is a schematic diagram illustrating the impact of the preprocessing module on the adaptive beamforming module 1 under different severe anomalies according to an embodiment of the present invention.

[0034] Figure 8 This is a schematic diagram of the anomaly simulation unit and anomaly simulation process in an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to specific embodiments:

[0036] An anomaly simulation method for a large-scale signal processing algorithm business chain, such as Figure 8 As shown, it includes the following steps:

[0037] Step 1: Based on the signal processing business chain testing requirements, identify the signal processing algorithm software that needs to be embedded with the anomaly simulation unit and the business chain links that need to run the anomaly simulation unit independently.

[0038] Step 2: Configure the anomaly simulation unit. According to the test requirements, enable the corresponding sub-units and initialize the behavior mode, utilization rate, number of occupied and other parameters of each sub-unit.

[0039] Step 3: Integrate the anomaly simulation unit and the signal processing algorithm software that needs to be embedded for testing into a container, and configure the corresponding running commands in the container; orchestrate the business chain for the anomaly simulation unit that needs to run independently; and register the information of the anomaly simulation unit to the business chain management center after completion.

[0040] Step 4: Based on the registration information, construct a directed graph for business chain anomaly testing, consisting of anomaly simulation units in the business chain. ; vertices of the graph This represents a container consisting of a signal processing algorithm software module and an anomaly simulation unit, or an independently running anomaly simulation unit; the vertex state is... It contains information such as the sub-units enabled by each anomaly simulation unit, the configuration of each sub-unit, and the status of the algorithm software; the edges of the graph This represents the impact of a change in the state of one vertex on the state of another vertex. In other words, it represents the impact of a module-level anomaly caused by the anomaly simulation unit on the software modules of each algorithm in the business chain. The initial state of the edge is unknown and is the result of the business chain testing process.

[0041] Among them, the vertex set , It represents the number of vertices. The set of edges. ,in Represents vertices State of Vertex The influence of state. The state of each vertex is represented by... It means that among them , , , , and These represent the states of the CPU resource usage subunit, GPU resource usage subunit, memory usage subunit, video memory usage subunit, traffic simulation subunit, and the embedded signal processing algorithm software module, respectively. If the anomaly simulation unit in a vertex runs independently within the business chain, then... The state is empty.

[0042] Step 5: Based on the established business chain anomaly test directed graph, generate multiple sets of anomaly states for each heterogeneous simulation unit. Each set of anomaly states includes different CPU core usage, CPU behavior patterns, CPU utilization, GPU resource usage, GPU behavior patterns, GPU utilization, video memory / memory behavior patterns, usage, video memory / memory bandwidth utilization, network behavior patterns, number of network ports occupied, and bandwidth utilization.

[0043] Step 6: Deploy the business chain to the actual hardware cluster, inject each group of abnormal states in sequence to start the test, record the vertex state changes caused by each group of abnormal states, and reset the entire business chain after each group of abnormal states is completed, and then proceed to the next group of abnormal states test.

[0044] Step 7: After completing the tests for all abnormal states, based on the recorded initial states of all vertices... and real-time status Estimate the edge states. During the directed graph testing process, the first... The vertex state change model for this test is as follows: ,in It represents the real-time state of all vertices. This is the initial state of all vertices. It is a matrix composed of edge states. Indicates the first There were [number] tests, with a total of [number] tests. Next. By solving the following equation

[0045] Calculate the estimated edge states This allows us to determine the extent to which each algorithm module affects other software modules when a module-level anomaly occurs.

[0046] The following will further illustrate this with specific scenarios:

[0047] This embodiment covers a signal service link with a total of 5 modules ( Figure 1 Anomalies are simulated to illustrate the effects of the invention. In this embodiment, the adaptive beamforming module uses CPU+GPU for computation, while other modules use only the CPU.

[0048] In this embodiment, each time a severe anomaly is injected, that is, each simulated anomaly value continuously occupies 100% of hardware resources. For example, if the preprocessing module occupies 30% of the CPU under normal conditions, when simulating a severe anomaly, the anomaly simulation unit will increase the CPU resource utilization rate to 100% and maintain it.

[0049] This embodiment uses the present invention to evaluate the mutual influence between various modules in the business chain. The degree of influence is categorized according to the degree of deviation from normal values: no influence (deviation less than 10%), slight (deviation between 10% and 50%), moderate (deviation between 50% and 100%), and severe (deviation greater than 100%). This is achieved through the edge state... The estimated results data can be visualized by anomaly type or by the impact between modules.

[0050] Figures 2 to 6 The first line represents the evaluation results of the mutual impact between modules in the business chain when different modules inject different types of severe anomalies. The second line represents the impact of each module itself, which is not included in the evaluation scope. Figure 2 This refers to the mutual influence between modules when a severe CPU anomaly occurs. Overall, an anomaly in an upstream module will affect a downstream module, while parallel modules will not affect each other. Figure 3 In the process, although the preprocessing module does not use a GPU, when the GPU of this module malfunctions, it will cause the downstream adaptive beamforming module, which uses a GPU, to malfunction. This is because the GPU of the device where the preprocessing module is located overheats, causing the entire machine to reduce its frequency, which slows down the data flow output by the module. Ultimately, this leads to a reduction in the computational load and GPU utilization of the adaptive beamforming module per unit time. Figure 4 Memory anomalies can cause corresponding anomalies in the data fusion module. This is because only two independent, parallel adaptive beamforming modules utilize the GPU. Figure 5 Anomalies caused by memory issues will not affect each other.

[0051] from Figures 2 to 6 It is evident that anomalies caused by CPU and network issues are most likely to propagate from upstream to downstream services.

[0052] Figure 7 This describes the impact of different types of anomalies on the adaptive beamforming module 1 in the preprocessing module. Aside from memory anomalies, other anomalies can cause more than one type of anomaly in the adaptive beamforming module.

[0053] On the other hand, the present invention provides an anomaly simulation unit for a large-scale signal processing algorithm business chain, including the following sub-units: CPU resource usage sub-unit, GPU resource usage sub-unit, memory usage sub-unit, video memory usage sub-unit, and traffic simulation sub-unit.

[0054] Each sub-unit can be enabled as needed according to testing requirements. Its main functions are as follows:

[0055] The CPU resource usage sub-unit is used to simulate the number of CPU cores, CPU behavior patterns, and utilization rates of various signal processing modules during runtime.

[0056] GPU resource usage sub-unit: used to simulate the number of GPUs used, GPU behavior patterns, and utilization rates for each module;

[0057] Memory usage sub-unit: used to simulate the memory behavior patterns, usage rate, and memory bandwidth utilization of each module;

[0058] Memory usage subunit: used to simulate the memory behavior patterns, memory usage rate, and memory bandwidth utilization of each module;

[0059] Traffic simulation subunit: used to simulate the network behavior patterns, number of network ports occupied, and bandwidth utilization of each module.

[0060] Before activation, the business chain anomaly simulation unit can perform batch initial configuration of the functions of its various sub-units. During runtime, it can perform batch remote dynamic configuration via independent network addresses to dynamically simulate specific anomalies.

[0061] This invention also provides an anomaly simulation unit device for a large-scale signal processing algorithm service chain, enabling the anomaly simulation unit to operate independently within the service chain. This device possesses communication interfaces for various physical links, including SRIO, gigabit Ethernet, 10-gigabit Ethernet, 25G optical ports, and 100G optical ports. The anomaly simulation unit device can additionally simulate various types of service chain data streams, working in conjunction with anomaly simulation units embedded with conventional algorithm software to complete service chain testing.

[0062] After receiving the initialization configuration information of the abnormal simulation unit, the abnormal simulation unit device can be integrated with other signal processing functions.

[0063] The above description only illustrates preferred embodiments of the present invention and should not be construed as limiting the scope of the claims. Any equivalent procedural modifications made using this specification are included within the patent protection scope of this invention.

Claims

1. A method for anomaly simulation in a large-scale signal processing algorithm business chain, characterized in that: Includes the following steps, Step 1: Based on the signal processing business chain testing requirements, identify the signal processing algorithm software that needs to be embedded with the anomaly simulation unit and the business chain links that need to run the anomaly simulation unit independently; Step 2: Configure the anomaly simulation unit, enable the corresponding sub-units, and initialize the behavior mode, utilization rate, and quantity parameters of each sub-unit; Step 3: Integrate the anomaly simulation unit and signal processing algorithm software into a container, and configure the running commands; Perform business chain orchestration on independently operating anomaly simulation units; register the anomaly simulation unit information to the business chain management center; Step 4: Based on the registration information, construct a directed graph for business chain anomaly testing, where vertices represent containers of anomaly simulation units and algorithm software modules or independently running anomaly simulation units, and edges represent the impact of a vertex state change on the state of another vertex. Step 5: Generate multiple sets of abnormal states for each abnormal simulation unit. Each set of abnormal states includes parameters such as CPU / GPU resource usage, memory / video memory usage, and network traffic usage. Step 6: Deploy the business chain to the hardware cluster, inject each group of abnormal states in sequence for testing, and record the changes in the vertex state; Step 7: Based on the recorded vertex state change data, estimate the edge state and evaluate the impact of each algorithm module on other modules under abnormal conditions.

2. The anomaly simulation method for a large-scale signal processing algorithm service chain according to claim 1, characterized in that: In step 4, the method for constructing the directed graph for business chain anomaly testing is represented as follows: Among them, the vertex set , This refers to a container consisting of a signal processing algorithm software module and an anomaly simulation unit, or an independently operating anomaly simulation unit. It is the number of vertices; edge set , Each element represents the influence of the state of vertex i on the CPU, GPU, memory, video memory, network, and software states of vertex j.

3. The anomaly simulation method for large-scale signal processing algorithm service chains according to claim 2, characterized in that: The state of each vertex is used It means that, among them , , , , and These represent the status of the CPU resource usage subunit, GPU resource usage subunit, memory usage subunit, video memory usage subunit, traffic simulation subunit, and the embedded signal processing algorithm software module, respectively.

4. The anomaly simulation method for a large-scale signal processing algorithm service chain according to claim 1, characterized in that: In step 7, the edge state estimation method is as follows: The first directed graph test process The vertex state change model for this test is as follows: ,in It represents the real-time state of all vertices. This is the initial state of all vertices. It is a matrix composed of edge states. Indicates the first There were [number] tests, with a total of [number] tests. Next, by solving the formula Calculate the estimated edge states This allows us to determine the extent to which each algorithm module affects other software modules when a module-level anomaly occurs.

5. An anomaly simulation unit for a large-scale signal processing algorithm service chain, characterized in that, include: The CPU resource usage subunit is used to simulate the number of CPU cores, behavior patterns, and utilization of the algorithm module. The GPU resource usage subunit is used to simulate the number of GPUs used, their behavior patterns, and utilization rates in the algorithm module. The memory usage subunit is used to simulate the memory behavior patterns, occupancy rate, and memory bandwidth utilization of the algorithm module. The memory usage subunit is used to simulate the memory behavior pattern, memory usage rate, and memory bandwidth utilization rate of the algorithm module. The traffic simulation subunit is used to simulate the network behavior patterns, network port occupancy, and bandwidth utilization of the algorithm module.

6. The anomaly simulation unit according to claim 5, characterized in that, Each sub-unit supports batch initial configuration and remote dynamic configuration, which can be enabled as needed according to testing requirements.

7. An anomaly simulation unit device, characterized in that, include: Multiple physical link communication interfaces, supporting SRIO, gigabit Ethernet, 10-gigabit Ethernet, 25G optical port and 100G optical port; The device is used to carry the anomaly simulation unit as described in claim 5, and supports its independent operation in the business chain.

8. The anomaly simulation unit device according to claim 7, characterized in that, The device can simulate various types of business chain data flows and work in conjunction with an anomaly simulation unit embedded with conventional algorithm software to complete the test.

Images