Convolutional neural network reasoning acceleration system and method based on heterogeneous architecture

Abstract

The invention discloses a convolutional neural network reasoning acceleration system and method based on a heterogeneous architecture, and relates to the technical field of network reasoning acceleration, and the method comprises the following steps: during the task scheduling period of a convolutional neural network, obtaining various parameter information generated in the task scheduling process of the convolutional neural network, wherein the parameters comprise task scheduling parameters and parallel processing parameters. By monitoring task scheduling and parallel processing parameters in real time, the system generates an abnormal scheduling index by using a machine learning model, and intelligently identifies and classifies whether the task state is normal or abnormal. And for anomalies, the anomalies are further subdivided into an accidental type or a non-accidental type, and corresponding treatment measures are taken. Non-accidental abnormity triggers early warning prompt, frequent intervention is avoided for accidental abnormity through continuous monitoring, fault processing accuracy and system stability are improved, efficient operation of a real-time monitoring system is ensured, and potential safety hazards are reduced.

Description

Technical Field

[0001] The present invention relates to the field of network inference acceleration technology, and in particular to a convolutional neural network inference acceleration system and method based on a heterogeneous architecture. Background Art

[0002] Convolutional neural network (CNN) inference acceleration based on heterogeneous architecture refers to improving the operating efficiency of CNN models in the inference stage by utilizing different types of hardware devices, such as CPU, GPU, FPGA or dedicated AI accelerators to work together. When processing high-dimensional data such as images and videos, convolutional neural networks usually involve a large number of matrix multiplications and convolution operations, which are computationally intensive and time-consuming. Heterogeneous architecture greatly improves the inference speed of the model by assigning different computing tasks to the most suitable hardware, such as assigning parts with strong parallel computing capabilities to GPUs and assigning low-latency operations to FPGAs. In addition, the parallelism and optimization algorithms of these hardware devices can reduce power consumption and latency during inference to meet real-time or near-real-time application requirements.

[0003] In practical applications, the core idea of ​​heterogeneous architecture inference acceleration is to make full use of the characteristics and advantages of various types of hardware to design efficient task scheduling and resource allocation strategies. The CNN inference stage is often performed in embedded devices or cloud services with limited resources. Therefore, heterogeneous architecture can maximize the utilization of computing resources through reasonable workload distribution. Modern AI inference acceleration platforms usually include highly optimized software stacks that support automatic mapping of different network components such as convolutional layers and fully connected layers to the most suitable hardware. This acceleration method can not only improve processing performance, but also reduce inference time while maintaining accuracy, thereby meeting AI application scenarios with high response speed requirements, such as autonomous driving, real-time monitoring, smart homes, etc.

[0004] The prior art has the following deficiencies:

[0005] Convolutional neural network (CNN) inference acceleration based on heterogeneous architecture plays a key role in real-time monitoring, significantly improving the efficiency of video processing and image recognition, ensuring that the system can respond quickly and make accurate decisions. However, CNN inference is highly dependent on efficient task scheduling, and different hardware devices need to coordinate and synchronize to complete computing tasks. If there is an abnormality in task scheduling and the system fails to detect it in time, it may cause a series of serious consequences, especially in real-time monitoring scenarios, which directly affects the reliability, efficiency and accuracy of the system's decisions.

[0006] Especially when multiple hardware devices need to coordinate and synchronize, abnormal task scheduling may cause deadlock or resource competition problems in the computing process. If some tasks are stuck in an infinite waiting state while waiting for other devices to complete synchronization, the entire reasoning process may be stuck and unable to continue. This will not only cause reasoning delays, but may also cause system crashes or downtime. In high-load, high-concurrency real-time monitoring scenarios, once the system crashes, the monitoring data will not be processed, causing the system to lose its monitoring function, and even in serious cases, the entire system may be paralyzed, posing a major security risk.

[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0008] The purpose of the present invention is to provide a convolutional neural network reasoning acceleration system and method based on a heterogeneous architecture. By real-time monitoring of task scheduling and parallel processing parameters, the system uses a machine learning model to generate an abnormal scheduling index, intelligently identify and classify task status as normal or abnormal. For abnormalities, they are further subdivided into accidental or non-accidental types, and corresponding treatment measures are taken. Non-accidental anomalies trigger early warning prompts, and accidental anomalies avoid frequent intervention through continuous monitoring, improve fault handling accuracy and system stability, ensure the efficient operation of the real-time monitoring system and reduce safety hazards, so as to solve the problems in the above-mentioned background technology.

[0009] In order to achieve the above object, the present invention provides the following technical solution: a convolutional neural network reasoning acceleration method based on a heterogeneous architecture, comprising the following steps:

[0010] During the convolutional neural network task scheduling, various parameter information generated during the convolutional neural network task scheduling process is obtained, including task scheduling parameters and parallel processing parameters;

[0011] Under the detection window, the acquired task scheduling parameters and parallel processing parameters are analyzed for abnormalities, and the task scheduling parameters and parallel processing parameters after abnormal analysis are input into the pre-learned machine model. The abnormal scheduling index is generated through the machine model, and the convolutional neural network task scheduling process is intelligently perceived based on the abnormal scheduling index.

[0012] Based on the results output by the machine learning model, namely the abnormal scheduling index, the convolutional neural network task scheduling process is divided into abnormal scheduling state or normal scheduling state;

[0013] For abnormal scheduling status, the operation information of the convolutional neural network task scheduling process is continuously obtained, and the abnormal scheduling status is further divided into accidental abnormality or non-accidental abnormality;

[0014] Different processing methods are adopted for different abnormal scheduling states.

[0015] Preferably, the task scheduling parameters include execution pipeline depth information, which is an important indicator in the task scheduling parameters. It determines how many steps the task is decomposed into for processing when executing the inference task on the hardware device. The parallel processing parameters include concurrency control information. The concurrency control information is the core indicator in the parallel processing parameters, which determines how many tasks can be processed simultaneously in the system.

[0016] Preferably, under the detection window, an abnormal analysis is performed on the execution pipeline depth information and the concurrency control information, and an execution pipeline depth index and a concurrency control index are generated respectively. The execution pipeline depth index and the concurrency control index are input into a pre-learned machine model, and an abnormal scheduling index is generated through the machine model. Based on the abnormal scheduling index, the convolutional neural network task scheduling process is intelligently perceived.

[0017] Preferably, the logic of performing abnormal analysis on the execution pipeline depth information and generating the execution pipeline depth index is as follows:

[0018] Under the detection window, obtain the execution pipeline depth data, and mark the obtained execution pipeline depth data as D i , where D i ={D 1 , D 2 , D 3 ,……,D n ,}, where D i Indicates the execution pipeline depth value at time point i, where n is the total number of time points;

[0019] In order to capture the degree of synchronization between the execution pipeline depth value and the hardware, the pipeline synchronization dependency coefficient is introduced, and the calculation expression is as follows:

[0020]

[0021] , where λ i is the pipeline synchronization dependency coefficient, which is used to measure the correlation strength between the execution pipeline depth value at the i-th time point and the synchronization with the rest of the devices. α is the sensitivity coefficient for controlling the pipeline depth difference. j Indicates the execution pipeline depth value at time point j;

[0022] As the pipeline depth changes, resource competition intensifies. Therefore, a resource competition conflict factor is introduced. The calculation expression is as follows:

[0023]

[0024] , where ρi is the resource competition conflict factor, C k is the current resource usage of the kth hardware device, R k is the total resource capacity of the kth device, m is the total number of hardware devices involved in executing the task, and β is the control coefficient of resource competition intensity;

[0025] Introducing the deadlock potential risk coefficient, the calculation expression is as follows:

[0026]

[0027] , where δ i represents the deadlock potential risk coefficient, which is used to evaluate the synchronization waiting risk caused by excessive pipeline depth at the i-th time point. γ is the deadlock sensitivity coefficient, which is used to control the impact of depth difference on deadlock risk.

[0028] Based on pipeline synchronization dependency coefficient λ i , resource competition conflict factor ρ i And the deadlock potential risk factor δ i , generate a comprehensive execution pipeline depth index, the calculation expression is as follows:

[0029]

[0030] , where EPDI represents the execution pipeline depth index, and θ is the execution pipeline depth index adjustment parameter, which is used to balance the impact of the depth value on the index.

[0031] Preferably, the logic of performing abnormal analysis on the concurrency control information and generating the concurrency control index is as follows:

[0032] In the detection window, we first evaluate the resources occupied by each task during execution, including memory, CPU, and communication bandwidth. The calculation expression is as follows:

[0033]

[0034] , where R e is the resource occupancy rate, which indicates the proportion of resources occupied by each task e in its life cycle, U m is the memory usage of task e, T cpu is the time that task e consumes on the CPU, U b is the occupancy rate of task e on the communication bandwidth, T comm is the communication time of task e, C max is the maximum resource capacity of the system, T total is the total execution time of task e;

[0035] Different tasks need to complete data exchange and collaborative execution synchronously. Synchronization overhead is an important factor in measuring concurrent task scheduling. The synchronization overhead calculation expression is as follows:

[0036]

[0037] , where S e is the synchronization overhead, which represents the ratio of the synchronization time required by task e to its total execution time, T sync It is the time that task e waits for other tasks or devices to complete synchronization during execution;

[0038] Another key factor in concurrency control is resource competition. When system resources are limited, competition will occur when multiple tasks compete for resources at the same time. The resource competition factor is introduced and the calculation expression is as follows:

[0039]

[0040] , where C f is the resource competition factor, T avail (s) represents the remaining amount of available resources in the system at time point s, and p is the total number of tasks;

[0041] Combined with the resource utilization rate R e , synchronization overhead S e and resource competition factor C f And the remaining amount of system available resources T avail (s) and calculate the system concurrent pressure. The calculation expression is as follows:

[0042]

[0043] , where P sys is the system concurrency pressure, T limit (s) is the maximum allowed execution time of each task in the system at time point s, which is related to the system load and scheduling strategy;

[0044] The concurrency control index is generated through comprehensive calculation of system concurrency pressure and resource competition factors. The calculation expression is as follows:

[0045]

[0046] , where CCI represents the concurrency control index.

[0047] Preferably, the execution pipeline depth index EPDI and the concurrency control index CCI are input into a pre-learned machine model, and the abnormal scheduling index AS is generated by the machine model, according to the following formula:

[0048]

[0049] , where V 1 、V 2 are the preset proportional coefficients of the execution pipeline depth index EPDI and the concurrency control index CCI, and V 1 、V 2 Both are greater than 0.

[0050] Preferably, the generated abnormal scheduling index is compared and analyzed with a preset abnormal scheduling index reference threshold, and the analysis result is as follows:

[0051] If the abnormal scheduling index is greater than or equal to a preset abnormal scheduling index reference threshold, the convolutional neural network task scheduling process is classified as an abnormal scheduling state;

[0052] If the abnormal scheduling index is less than the preset abnormal scheduling index reference threshold, the convolutional neural network task scheduling process is divided into a normal scheduling state.

[0053] Preferably, when an abnormal scheduling state is generated during the convolutional neural network task scheduling process, a number of abnormal scheduling indices generated subsequently are comprehensively analyzed with a preset abnormal scheduling index reference threshold to generate a scheduling evaluation index DD, based on the formula: Where, AS r represents an abnormal scheduling index that is greater than or equal to an abnormal scheduling index reference threshold, r represents the number of the abnormal scheduling index that is greater than or equal to the abnormal scheduling index reference threshold, r=1, 2, 3..., h, and h is a positive integer.

[0054] The generated dispatch evaluation index is compared and analyzed with the preset dispatch evaluation index reference threshold, and the analysis results are as follows:

[0055] If the scheduling evaluation index is greater than or equal to the preset scheduling evaluation index reference threshold, the convolutional neural network task scheduling process is classified as a non-accidental abnormality;

[0056] If the scheduling evaluation index is less than the preset scheduling evaluation index reference threshold, the convolutional neural network task scheduling process is classified as an accidental abnormality.

[0057] The convolutional neural network inference acceleration system based on heterogeneous architecture includes parameter acquisition module, abnormal analysis and perception module, scheduling status judgment module, abnormal type identification module and abnormal handling module:

[0058] A parameter acquisition module, during the convolutional neural network task scheduling, obtains various parameter information generated during the convolutional neural network task scheduling process, including task scheduling parameters and parallel processing parameters;

[0059] The abnormality analysis and perception module performs abnormality analysis on the acquired task scheduling parameters and parallel processing parameters under the detection window, and inputs the task scheduling parameters and parallel processing parameters after abnormality analysis into the pre-learned machine model, generates abnormal scheduling index through the machine model, and performs intelligent perception of the convolutional neural network task scheduling process based on the abnormal scheduling index;

[0060] The scheduling state judgment module divides the convolutional neural network task scheduling process into abnormal scheduling state or normal scheduling state based on the output of the machine learning model, namely the abnormal scheduling index;

[0061] The abnormal type identification module continuously obtains the operation information of the convolutional neural network task scheduling process for the abnormal scheduling state, and further divides the abnormal scheduling state into accidental abnormality or non-accidental abnormality;

[0062] The exception handling module adopts different handling methods for different abnormal scheduling states

[0063] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0064] The present invention can intelligently perceive the task scheduling state of the convolutional neural network through real-time monitoring and analysis of task scheduling parameters and parallel processing parameters. By generating an abnormal scheduling index through a machine learning model, the system can timely identify and classify the state of task scheduling, and divide it into a normal scheduling state or an abnormal scheduling state. Once an abnormal scheduling state is detected, the system will further analyze and refine the abnormal scheduling into accidental abnormalities or non-accidental abnormalities. In this way, not only can abnormal situations be accurately captured, reasoning delays, system crashes or downtime caused by problems such as deadlocks and resource competition be avoided, but also appropriate countermeasures can be taken according to different abnormal types to ensure the long-term stability and efficient operation of the system. Especially in high-load, high-concurrency real-time monitoring scenarios, this intelligent perception and classification can prevent the monitoring system from being paralyzed due to abnormalities that are not handled in a timely manner, and reduce the occurrence of major safety hazards.

[0065] The present invention proposes a flexible processing method for different types of abnormalities. For non-accidental abnormalities, the system can issue early warning prompts in time to remind relevant staff to perform maintenance to prevent further system damage or performance degradation. For accidental abnormalities, the system analyzes the conditions and frequency of their occurrence through continuous monitoring and recording, avoids frequent intervention, and takes long-term optimization measures when necessary. This classification processing method based on the scheduling evaluation index not only improves the accuracy of fault handling, but also avoids excessive maintenance due to occasional abnormalities, saving resources and time. In addition, by comprehensively analyzing the abnormal scheduling index generated subsequently, the system can determine whether more in-depth intervention is needed to ensure that the real-time monitoring system can self-recover or be effectively optimized in a short time, thereby enhancing the adaptability and flexibility of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0067] Figure 1 This is a flow chart of the method for accelerating convolutional neural network reasoning based on heterogeneous architecture of the present invention.

[0068] Figure 2 This is a module schematic diagram of the convolutional neural network reasoning acceleration system based on heterogeneous architecture of the present invention. DETAILED DESCRIPTION

[0069] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0070] The present invention provides Figure 1 The convolutional neural network reasoning acceleration method based on heterogeneous architecture shown includes the following steps:

[0071] During the convolutional neural network task scheduling, various parameter information generated during the convolutional neural network task scheduling process is obtained, including task scheduling parameters and parallel processing parameters;

[0072] Task scheduling parameters include execution pipeline depth information, which is an important indicator in task scheduling parameters. It determines how many steps the task is decomposed into for processing when executing the inference task on the hardware device. Parallel processing parameters include concurrency control information, which is the core indicator in parallel processing parameters. It determines how many tasks can be processed simultaneously in the system.

[0073] Under the detection window, the acquired task scheduling parameters and parallel processing parameters are analyzed for abnormalities, and the task scheduling parameters and parallel processing parameters after abnormal analysis are input into the pre-learned machine model. The abnormal scheduling index is generated through the machine model, and the convolutional neural network task scheduling process is intelligently perceived based on the abnormal scheduling index.

[0074] Under the detection window, perform abnormal analysis on the execution pipeline depth information and concurrency control information, generate the execution pipeline depth index and concurrency control index respectively, and input the execution pipeline depth index and concurrency control index into the pre-learned machine model, generate the abnormal scheduling index through the machine model, and perform intelligent perception of the convolutional neural network task scheduling process based on the abnormal scheduling index;

[0075] In the task scheduling process of convolutional neural networks (CNNs), changes in the execution pipeline depth index may cause problems in the decomposition and synchronization of computing tasks on different hardware devices. If the execution pipeline is too deep, the tasks are over-subdivided, resulting in frequent synchronization and data transmission requirements between hardware devices, increasing the synchronization overhead. When some tasks need to wait for other devices to complete part of the task before continuing to execute, synchronization delays or even deadlocks may occur. When deadlock occurs, multiple tasks wait for each other to release resources, causing the system to fall into an infinite waiting state, and the reasoning process is stuck and cannot continue to execute. This will lead to increased reasoning delays, especially in high-load and high-concurrency real-time monitoring environments, where the entire system may crash or shut down due to resource competition or overload. Once the system crashes, the monitoring data will not be processed in time, resulting in the loss of monitoring functions and the inability to monitor key events. It may even affect the normal operation of the entire monitoring system, bringing serious safety hazards and potential major risks. Therefore, changes in the execution pipeline depth index must be carefully managed to avoid these problems.

[0076] The logic of performing abnormal analysis on the execution pipeline depth information and generating the execution pipeline depth index is as follows:

[0077] Under the detection window, obtain the execution pipeline depth data, and mark the obtained execution pipeline depth data as D i , where D i ={D 1 , D 2 ,d3 ,……,D n ,}, where D i Indicates the execution pipeline depth value at time point i, where n is the total number of time points;

[0078] It should be noted that the execution pipeline depth data refers to the data related to the decomposition and processing steps of each task in the hardware device during the execution of the convolutional neural network (CNN) inference task. This data helps us understand how tasks are split, synchronized, and executed in the pipeline.

[0079] Specific execution pipeline depth data may include the following aspects:

[0080] 1. Execution depth data for each stage

[0081] This refers to the degree of decomposition and number of execution steps of each stage of the inference task pipeline. This data is used to track the execution progress of the task on the hardware device and usually includes the following information:

[0082] Number of task stages: Indicates how many different execution stages the inference task is divided into in the pipeline. This value affects the way the hardware device processes tasks and is usually proportional to the depth of the pipeline.

[0083] The amount of tasks in each stage: Indicates the amount of data or computation required to be processed in each stage. These data are used to understand the distribution of tasks in each stage to ensure load balance.

[0084] 2. Synchronization waiting time

[0085] Synchronization wait time refers to the synchronization time between different hardware devices or tasks during the execution of pipeline depth. The execution pipeline depth data will record:

[0086] Synchronization wait time: It is used to understand the time a task or hardware device waits for other tasks to complete. If the synchronization wait time is too long, it may cause system delays or waste of resources.

[0087] Synchronization times: used to understand the synchronization frequency between different stages during the execution of the inference task. Frequent synchronization increases system overhead, while too little synchronization may lead to data inconsistency.

[0088] 3. Data transfer volume

[0089] During the execution of the pipeline depth, intermediate data needs to be continuously transmitted between devices. The amount of data transmission is closely related to the depth of the pipeline. The greater the depth, the more times and amount of intermediate data are transmitted. The relevant data includes:

[0090] The amount of data transmitted at each stage: indicates the size of data that the device needs to transfer between each pipeline stage. If the transmitted data is too large, it may cause bandwidth bottlenecks and affect the overall performance of the system.

[0091] Transmission delay: records the delay time of each data transmission, which is used to evaluate the efficiency of data transmission between devices. High latency means that the execution pipeline depth design is unreasonable, affecting the task execution speed.

[0092] 4. Resource usage data

[0093] During the execution of pipeline depth, hardware resources (such as memory, CPU, GPU, etc.) will be occupied by tasks at different stages. Recording resource usage data helps optimize pipeline depth settings:

[0094] Memory usage: used to monitor the amount of hardware memory required by the pipeline depth. A deeper pipeline may lead to increased memory usage because the intermediate data of each stage needs to be stored.

[0095] Hardware computing resource utilization: Record the usage of hardware devices (such as GPU utilization) in each pipeline stage. This helps evaluate whether the pipeline design can effectively utilize hardware resources.

[0096] 5. Task allocation and execution time

[0097] The execution pipeline depth data also includes the execution time of each task assigned to a specific hardware device. The relevant data includes:

[0098] Task allocation strategy: records how tasks are allocated on heterogeneous hardware devices (such as CPU, GPU, FPGA) in pipeline depth design. This data is used to analyze the impact of pipeline depth on hardware utilization and load balancing.

[0099] Execution time of each stage: Indicates the actual execution time of tasks on different devices in each pipeline stage. These data are used to evaluate whether the decomposition of the pipeline is reasonable and whether there is a bottleneck in the execution of tasks.

[0100] 6. Task completion time and concurrency

[0101] The execution pipeline depth data also includes task completion and concurrency information:

[0102] Single task completion time: records the time required for tasks in each pipeline stage from start to completion, which is used to analyze the response speed of the system.

[0103] Number of concurrent tasks: Indicates the number of tasks that the system processes simultaneously at each stage of pipeline execution. Concurrency affects hardware load and resource competition.

[0104] 7. Abnormal status and error log

[0105] In the actual pipeline execution process, abnormal status and error logs are also important in-depth data. This information includes:

[0106] Deadlock detection: Records whether the system has deadlocked during execution. When a task waits for other tasks for too long, this data can help detect potential deadlock problems.

[0107] Resource contention conflict: used to track whether resource contention problems occur during the execution process. When multiple tasks compete for the same resources, the execution efficiency of the system will be affected.

[0108] The execution pipeline depth data contains information from task decomposition, synchronization, resource usage to execution time, anomaly detection, and other aspects. These data help system designers evaluate the rationality of pipeline depth to ensure efficient resource utilization and reasonable task parallelism during task scheduling, while avoiding performance bottlenecks or crashes caused by synchronization problems or resource competition. These data provide a basis for system optimization, especially in high-load real-time monitoring scenarios.

[0109] In order to capture the degree of synchronization between the execution pipeline depth value and the hardware, the pipeline synchronization dependency coefficient is introduced, and the calculation expression is as follows:

[0110]

[0111] , where λ i is the pipeline synchronization dependency coefficient, which is used to measure the correlation strength between the execution pipeline depth value at the i-th time point and the synchronization with the rest of the devices. α is the sensitivity coefficient for controlling the pipeline depth difference. j Indicates the execution pipeline depth value at time point j;

[0112] The execution pipeline depth and the tightness of hardware synchronization refer to the degree of synchronization and coordination between the various stages of the task when different hardware devices work together according to the steps of the pipeline decomposition during the task execution process. The tighter the synchronization, the more efficient the tasks of each stage can be completed in sequence, the shorter the waiting time between hardware, and the smoother the overall system operation; on the contrary, if the synchronization is not tight enough, the waiting time of tasks between different devices will increase, which may lead to resource waste or system delays, affecting execution efficiency.

[0113] The pipeline synchronization dependency coefficient is an indicator used to measure the degree of synchronization and coordination between different hardware devices during task execution. It reflects the degree of dependence between different devices at each stage when executing pipeline tasks, that is, when a device executes a task, it needs to wait for other devices to complete the synchronization task. If the synchronization dependency is strong, it means that the task coordination requirements between the devices are high, and the system synchronization waiting time may be long; if the synchronization dependency is weak, it means that each task can be processed in parallel relatively independently, and the system efficiency is higher.

[0114] Why introduce pipeline synchronization dependency coefficient:

[0115] 1. Evaluate the synchronization efficiency of the system: In a heterogeneous hardware architecture, when multiple devices execute inference tasks in parallel, if the synchronization dependency is large, the difficulty of task coordination between devices increases. The synchronization dependency coefficient helps us understand the coordination requirements between tasks, thereby optimizing the task scheduling and synchronization strategy between devices and reducing waiting time.

[0116] 2. Avoid system bottlenecks: When tasks have strong synchronization dependencies, a device may be idle for a long time while waiting for other devices to complete, resulting in system performance degradation. Introducing synchronization dependency coefficients can identify these potential bottlenecks in advance and optimize resource allocation.

[0117] 3. Optimize pipeline design: By introducing the pipeline synchronization dependency coefficient, it can help design a more reasonable pipeline depth configuration to ensure that tasks can be efficiently allocated and executed between different devices, improve overall parallel efficiency, and avoid synchronization conflicts and deadlocks.

[0118] In short, the pipeline synchronization dependency coefficient is used to quantify the synchronization and coordination requirements between devices, help optimize task scheduling and synchronization strategies, and improve system performance and parallel processing efficiency.

[0119] As the pipeline depth changes, resource competition intensifies. Therefore, a resource competition conflict factor is introduced. The calculation expression is as follows:

[0120]

[0121] , where ρ i is the resource competition conflict factor, ρ i The closer it is to 0, the more intense the resource competition is; the closer it is to 1, the lower the resource competition is. k is the current resource usage of the kth hardware device, R k is the total resource capacity of the kth device, m is the total number of hardware devices involved in executing the task, and β is the control coefficient of resource competition intensity;

[0122] Why pipeline depth changes increase resource contention

[0123] As the pipeline depth increases, tasks are broken down into more subtasks, and multiple hardware devices need to process more task stages simultaneously. This results in an increase in the demand for limited hardware resources (such as memory, computing units, data bandwidth, etc.) from multiple tasks at the same time. For example, devices such as GPUs and CPUs require more computing resources to process the decomposed subtasks, and the resources of these devices are limited. When multiple tasks request these resources at the same time, resource availability becomes tight and competition intensifies. If resources are not allocated properly, task execution will be blocked and overall system performance will degrade.

[0124] What are resource competition conflict factors?

[0125] Resource Conflict Factor is an indicator used to quantify the degree of competition between hardware devices for limited resources (such as memory, bandwidth, computing cores, etc.) during task execution. It measures the intensity of competition by evaluating the intensity of demand for the same resources by multiple tasks in the system. The larger the resource conflict factor, the more intense the competition for resources among multiple tasks. As a result, bottlenecks may occur in the system, resulting in reduced task execution efficiency, and even delays or deadlocks.

[0126] The role of introducing resource competition conflict factors

[0127] The main purpose of introducing resource competition conflict factors is to help system designers and operation and maintenance personnel identify possible performance bottlenecks by quantifying resource competition. By analyzing the competition conflict factors, the system can optimize the resource allocation of tasks, avoid multiple tasks competing for the same hardware resources at the same time, and reduce the intensity of resource competition. It can also provide a basis for dynamic adjustment of the system, so that tasks can be more efficiently scheduled between different devices, thereby improving the overall parallel processing capability and stability of the system and reducing the risk of delays and conflicts.

[0128] Introducing the deadlock potential risk coefficient, the calculation expression is as follows:

[0129]

[0130] , where δ i represents the deadlock potential risk coefficient, which is used to evaluate the synchronization waiting risk caused by excessive pipeline depth at the i-th time point. γ is the deadlock sensitivity coefficient, which is used to control the impact of depth difference on deadlock risk.

[0131] The deadlock potential risk factor is an indicator used to measure the possibility of deadlock in the system during task execution. It analyzes the dependencies and synchronization between tasks on different devices to evaluate whether there are multiple tasks waiting for resources or completing synchronously during pipeline execution. When the dependencies between tasks are too strong or the synchronization requirements are high, the deadlock risk increases. This factor can help identify the possible deadlock risks in the system and provide a basis for optimizing task scheduling and resource allocation to avoid indefinite blocking of task execution.

[0132] Based on pipeline synchronization dependency coefficient λ i , resource competition conflict factor ρ i And the deadlock potential risk factor δ i , generate a comprehensive execution pipeline depth index, the calculation expression is as follows:

[0133]

[0134] , where EPDI represents the execution pipeline depth index, and θ is the execution pipeline depth index adjustment parameter, which is used to balance the impact of the depth value on the index;

[0135] It can be seen from the calculation expression of the execution pipeline depth index that the larger the execution pipeline depth index performance value generated after the abnormal analysis of the execution pipeline depth information, it usually indicates that there are problems with the decomposition and synchronization of tasks in the calculation process, increasing the risk of deadlock or resource competition. When a task is stuck in an infinite waiting state because it is waiting for other devices to complete synchronization, the reasoning process will be stuck, causing the system to experience reasoning delays or even crash or downtime. In a high-load, high-concurrency real-time monitoring environment, a system crash will cause the monitoring data to be unable to be processed, the monitoring function to fail, and in severe cases, it may even cause the entire system to paralyze, posing a major safety hazard. Therefore, the larger the execution pipeline depth index performance value, the higher the probability of system crash or paralysis; the smaller the execution pipeline depth index performance value, the lower the probability of such problems occurring in the system.

[0136] During the convolutional neural network (CNN) task scheduling process, if the concurrency control index changes, it may lead to intensified competition for computing resources and even cause deadlock or resource competition problems. In a high-concurrency real-time monitoring scenario, the system needs to coordinate multiple hardware devices to process tasks simultaneously. If the concurrency control index is set too high, too many tasks will run at the same time, and the hardware resources may be over-occupied, causing some tasks to fall into an infinite waiting state due to waiting for resources or device synchronization, thereby causing deadlock. In addition, excessive concurrency will increase the context switching overhead, resulting in reduced task scheduling efficiency and delayed reasoning process. Since the CNN reasoning process relies on the sequential nature of multi-layer calculations, once a task cannot be completed, its subsequent tasks cannot continue to execute, which will stagnate the entire reasoning process. In a high-load environment, excessive pressure on system resources may cause the system to crash or shut down, further making it impossible to process real-time monitoring data, causing the system to lose function, and even in severe cases, the entire monitoring system may be completely paralyzed, posing serious security risks.

[0137] The logic of performing abnormal analysis on concurrency control information and generating concurrency control index is as follows:

[0138] In the detection window, we first evaluate the resources occupied by each task during execution, including memory, CPU, and communication bandwidth. The calculation expression is as follows:

[0139]

[0140] , where R e is the resource occupancy rate, which indicates the proportion of resources occupied by each task e in its life cycle, U m is the memory usage of task e, T cpu is the time that task e consumes on the CPU, U b is the occupancy rate of task e on the communication bandwidth, T comm is the communication time of task e (i.e., the time it takes for the task to exchange data between hardware), C max is the maximum resource capacity of the system, T total is the total execution time of task e;

[0141] Calculating the resources occupied by each task during execution is to optimize the system's performance and resource utilization efficiency. By understanding the resource requirements of the task, we can ensure the reasonable allocation of resources and avoid resource overload or imbalance, thereby reducing computing bottlenecks, improving parallel processing efficiency, and avoiding deadlocks and resource competition. In addition, this helps predict system load in a high-concurrency environment, ensure the smooth execution of real-time tasks, reduce system latency, and improve overall stability and response speed.

[0142] Different tasks need to complete data exchange and collaborative execution synchronously. Synchronization overhead is an important factor in measuring concurrent task scheduling. The synchronization overhead calculation expression is as follows:

[0143]

[0144] , where S P is the synchronization overhead, which represents the ratio of the synchronization time required by task e to its total execution time, T sync It is the time that task e waits for other tasks or devices to complete synchronization during execution;

[0145] Synchronization overhead refers to the time and resources consumed by multiple tasks or threads in a concurrent system to maintain data consistency or coordinate the execution order during execution. It usually includes the waiting time for tasks to acquire locks for shared resources, the communication time for data transfer, and the collaborative processing time between devices or threads. The increase in synchronization overhead will cause task execution delays and reduce the overall efficiency of the system. Especially in high-concurrency systems, excessive synchronization overhead may cause performance bottlenecks or resource competition.

[0146] Another key factor in concurrency control is resource competition. When system resources are limited, competition will occur when multiple tasks compete for resources at the same time. The resource competition factor is introduced and the calculation expression is as follows:

[0147]

[0148] , where C f is the resource competition factor, T avail (s) represents the remaining amount of available resources in the system at time point s, and p is the total number of tasks;

[0149] Combined with the resource utilization rate R e , synchronization overhead S e and resource competition factor C f And the remaining amount of system available resources T avail (s) and calculate the system concurrent pressure. The calculation expression is as follows:

[0150]

[0151] , where P sys is the system concurrency pressure, T limit (s) is the maximum allowed execution time of each task in the system at time point s, which is related to the system load and scheduling strategy;

[0152] System concurrency pressure refers to the load caused by the consumption and competition of resources (such as CPU, memory, I / O bandwidth, etc.) when the system processes multiple concurrent tasks at the same time. It measures the load that the system bears in a high-concurrency environment, including resource usage, synchronization waiting, scheduling overhead, etc. when tasks are executed. When the concurrency pressure is too high, the system may face performance degradation, increased response time, fierce resource competition, and even cause task delays, deadlocks or crashes. Therefore, system concurrency pressure is an important indicator to measure the system's concurrent processing capabilities and resource management efficiency.

[0153] T limit (s) is the maximum allowed execution time of each task in the system at time point s, which is usually obtained in the following way:

[0154] 1. System Configuration

[0155] In some systems, T limit (s) may be a configuration parameter pre-set by a system administrator or engineer. The value can be manually set according to the needs of the application scenario (such as real-time requirements). For example, in a real-time monitoring system, the maximum execution time of a task can be configured according to the response requirements of the system.

[0156] 2. Task priority and scheduling strategy

[0157] T limit (s) may also be dynamically determined by the system's scheduling policy. Based on the task priority and the current system load, the scheduler assigns an upper limit on the allowed execution time to each task. High-priority tasks may be assigned shorter T limit (s) to ensure fast execution, while low-priority tasks may get longer execution times.

[0158] 3. Real-time system constraints

[0159] In hard real-time systems, T limit (s) is defined by real-time constraints, that is, the task must be completed within a specific time limit. This is usually directly related to the real-time requirements of the system. The scheduler will ensure that the task is completed within the specified time by monitoring the execution progress of the task and the system resources.

[0160] 4. Dynamic adjustment based on system load

[0161] T limit (s) may also be adjusted dynamically based on the current system load and resource utilization. The system will adjust the maximum execution time of each task in real time based on factors such as the number of concurrent tasks, resource availability, and synchronization overhead. When the system load is light, T limit (s) may increase accordingly; when the system load is heavy, T limit(s) may be reduced to prevent the task from taking up too many resources and affecting the overall system performance.

[0162] 5. Task historical execution time analysis

[0163] The system may automatically generate T based on the historical execution of the task limit If a certain type of task historically typically takes a certain amount of time to complete, the system can assign a reasonable maximum execution time to future tasks based on this data to avoid unnecessary delays.

[0164] In summary, T limit (s) are obtained through various means such as system configuration, scheduling strategy, real-time constraints, dynamic load management and task history analysis, with the aim of ensuring that tasks are completed efficiently and orderly in a concurrent environment.

[0165] The concurrency control index is generated through comprehensive calculation of system concurrency pressure and resource competition factors. The calculation expression is as follows:

[0166]

[0167] , where CCI represents the concurrency control index;

[0168] From the concurrency control index calculation expression, it can be seen that the larger the concurrency control index performance value generated after abnormal analysis of concurrency control information, the more concurrent tasks there are in the system under the detection window, which increases the risk of resource competition and task synchronization delay. This may cause deadlock or resource competition problems in the calculation process, causing some tasks to fall into an infinite waiting state due to waiting for other devices to complete synchronization, causing the entire reasoning process to stagnate. As the concurrency control index increases, the probability of reasoning delay, system crash or downtime also increases, especially in high-load, high-concurrency real-time monitoring scenarios. This situation is more serious and may eventually lead to the inability to process monitoring data, and even cause system paralysis and major security risks. On the contrary, if the concurrency control index performance value is smaller, the system resource competition is reduced, and the probability of deadlock or downtime will also be reduced.

[0169] The execution pipeline depth index EPDI and the concurrency control index CCI are input into the pre-learned machine model, and the abnormal scheduling index AS is generated through the machine model. The formula is as follows:

[0170]

[0171] , where V 1 、V 2 are the preset proportional coefficients of the execution pipeline depth index EPDI and the concurrency control index CCI, and V 1 、V 2Both are greater than 0.

[0172] It can be seen from the calculation expression of the abnormal scheduling index that the larger the performance value of the execution pipeline depth index generated after the abnormal analysis of the execution pipeline depth information, the larger the performance value of the concurrency control index generated after the abnormal analysis of the concurrency control information, that is, the larger the performance value of the abnormal scheduling index, the greater the probability of an abnormality in the convolutional neural network task scheduling process, and vice versa, the smaller the probability of an abnormality in the convolutional neural network task scheduling process.

[0173] Based on the results output by the machine learning model, namely the abnormal scheduling index, the convolutional neural network task scheduling process is divided into abnormal scheduling state or normal scheduling state;

[0174] The generated abnormal scheduling index is compared with the preset abnormal scheduling index reference threshold, and the analysis results are as follows:

[0175] If the abnormal scheduling index is greater than or equal to a preset abnormal scheduling index reference threshold, the convolutional neural network task scheduling process is classified as an abnormal scheduling state;

[0176] If the abnormal scheduling index is less than the preset abnormal scheduling index reference threshold, the convolutional neural network task scheduling process is divided into a normal scheduling state;

[0177] For abnormal scheduling status, the operation information of the convolutional neural network task scheduling process is continuously obtained, and the abnormal scheduling status is further divided into accidental abnormality or non-accidental abnormality;

[0178] When an abnormal scheduling state is generated during the convolutional neural network task scheduling process, a number of abnormal scheduling indices generated subsequently are comprehensively analyzed with the pre-set abnormal scheduling index reference threshold to generate a scheduling evaluation index DD, based on the formula: Where, AS r represents an abnormal scheduling index that is greater than or equal to an abnormal scheduling index reference threshold, r represents the number of the abnormal scheduling index that is greater than or equal to the abnormal scheduling index reference threshold, r=1, 2, 3..., h, and h is a positive integer.

[0179] The generated dispatch evaluation index is compared and analyzed with the preset dispatch evaluation index reference threshold, and the analysis results are as follows:

[0180] If the scheduling evaluation index is greater than or equal to the preset scheduling evaluation index reference threshold, the convolutional neural network task scheduling process is classified as a non-accidental abnormality;

[0181] If the scheduling evaluation index is less than the preset scheduling evaluation index reference threshold, the convolutional neural network task scheduling process is classified as an accidental abnormality;

[0182] Different processing methods are adopted for different abnormal scheduling states;

[0183] For non-accidental anomalies, an early warning will be issued to remind relevant staff that abnormalities have occurred in the convolutional neural network task scheduling process and to perform maintenance in a timely manner. For accidental anomalies, given that they may recover in a short period of time, no early warning information will be issued, but accidental anomalies should be monitored and recorded, and the conditions and frequency of their occurrence should be analyzed so that longer-term optimization measures can be taken when necessary.

[0184] The present invention can intelligently perceive the task scheduling state of the convolutional neural network through real-time monitoring and analysis of task scheduling parameters and parallel processing parameters. By generating an abnormal scheduling index through a machine learning model, the system can timely identify and classify the state of task scheduling, and divide it into a normal scheduling state or an abnormal scheduling state. Once an abnormal scheduling state is detected, the system will further analyze and refine the abnormal scheduling into accidental abnormalities or non-accidental abnormalities. In this way, not only can abnormal situations be accurately captured, reasoning delays, system crashes or downtime caused by problems such as deadlocks and resource competition be avoided, but also appropriate countermeasures can be taken according to different abnormal types to ensure the long-term stability and efficient operation of the system. Especially in high-load, high-concurrency real-time monitoring scenarios, this intelligent perception and classification can prevent the monitoring system from being paralyzed due to abnormalities that are not handled in a timely manner, and reduce the occurrence of major safety hazards.

[0185] The present invention proposes a flexible processing method for different types of abnormalities. For non-accidental abnormalities, the system can issue early warning prompts in time to remind relevant staff to perform maintenance to prevent further system damage or performance degradation. For accidental abnormalities, the system analyzes the conditions and frequency of their occurrence through continuous monitoring and recording, avoids frequent intervention, and takes long-term optimization measures when necessary. This classification processing method based on the scheduling evaluation index not only improves the accuracy of fault handling, but also avoids excessive maintenance due to occasional abnormalities, saving resources and time. In addition, by comprehensively analyzing the abnormal scheduling index generated subsequently, the system can determine whether more in-depth intervention is needed to ensure that the real-time monitoring system can self-recover or be effectively optimized in a short time, thereby enhancing the adaptability and flexibility of the overall system.

[0186] The present invention provides Figure 2 The convolutional neural network inference acceleration system based on heterogeneous architecture shown in the figure includes a parameter acquisition module, an abnormality analysis and perception module, a scheduling state judgment module, an abnormality type recognition module and an abnormality handling module:

[0187] A parameter acquisition module, during the convolutional neural network task scheduling, obtains various parameter information generated during the convolutional neural network task scheduling process, including task scheduling parameters and parallel processing parameters;

[0188] The abnormality analysis and perception module performs abnormality analysis on the acquired task scheduling parameters and parallel processing parameters under the detection window, and inputs the task scheduling parameters and parallel processing parameters after abnormality analysis into the pre-learned machine model, generates abnormal scheduling index through the machine model, and performs intelligent perception of the convolutional neural network task scheduling process based on the abnormal scheduling index;

[0189] The scheduling state judgment module divides the convolutional neural network task scheduling process into abnormal scheduling state or normal scheduling state based on the output of the machine learning model, namely the abnormal scheduling index;

[0190] The abnormal type identification module continuously obtains the operation information of the convolutional neural network task scheduling process for the abnormal scheduling state, and further divides the abnormal scheduling state into accidental abnormality or non-accidental abnormality;

[0191] The exception handling module adopts different processing methods for different abnormal scheduling states.

[0192] The convolutional neural network reasoning acceleration method based on heterogeneous architecture provided in the embodiment of the present invention is implemented by the convolutional neural network reasoning acceleration system based on heterogeneous architecture. The specific method and process of the convolutional neural network reasoning acceleration system based on heterogeneous architecture are detailed in the embodiment of the convolutional neural network reasoning acceleration method based on heterogeneous architecture, which will not be repeated here.

[0193] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A convolutional neural network reasoning acceleration method based on heterogeneous architecture, characterized in that: The following steps are involved: During the convolutional neural network task scheduling, various parameter information generated during the convolutional neural network task scheduling process is obtained, including task scheduling parameters and parallel processing parameters; Under the detection window, the acquired task scheduling parameters and parallel processing parameters are analyzed for abnormalities, and the task scheduling parameters and parallel processing parameters after abnormal analysis are input into the pre-learned machine model. The abnormal scheduling index is generated through the machine model, and the convolutional neural network task scheduling process is intelligently perceived based on the abnormal scheduling index. Based on the results output by the machine learning model, namely the abnormal scheduling index, the convolutional neural network task scheduling process is divided into abnormal scheduling state or normal scheduling state; For abnormal scheduling status, the operation information of the convolutional neural network task scheduling process is continuously obtained, and the abnormal scheduling status is further divided into accidental abnormality or non-accidental abnormality; Different processing methods are adopted for different abnormal scheduling states.

2. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 1, characterized in that: Task scheduling parameters include execution pipeline depth information. Execution pipeline depth information is an important indicator in task scheduling parameters. It determines how many steps the task is broken down into for processing when executing inference tasks on hardware devices. Parallel processing parameters include concurrency control information. Concurrency control information is the core indicator in parallel processing parameters. It determines how many tasks can be processed simultaneously in the system.

3. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 2, characterized in that: Under the detection window, the execution pipeline depth information and concurrency control information are analyzed for abnormalities, and the execution pipeline depth index and concurrency control index are generated respectively. The execution pipeline depth index and concurrency control index are input into the pre-learned machine model, and the abnormal scheduling index is generated through the machine model. Based on the abnormal scheduling index, the convolutional neural network task scheduling process is intelligently perceived.

4. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 3, characterized in that: The logic of performing abnormal analysis on the execution pipeline depth information and generating the execution pipeline depth index is as follows: Under the detection window, obtain the execution pipeline depth data, and mark the obtained execution pipeline depth data as D i , where D i ={D1, D2, D3, ..., D n ,}, where D i Indicates the execution pipeline depth value at time point i, where n is the total number of time points; In order to capture the degree of synchronization between the execution pipeline depth value and the hardware, the pipeline synchronization dependency coefficient is introduced, and the calculation expression is as follows: In the formula, λ i is the pipeline synchronization dependency coefficient, which is used to measure the correlation strength between the execution pipeline depth value at the i-th time point and the synchronization with the rest of the devices. α is the sensitivity coefficient for controlling the pipeline depth difference. j Indicates the execution pipeline depth value at time point j; As the pipeline depth changes, resource competition intensifies. Therefore, a resource competition conflict factor is introduced. The calculation expression is as follows: In the formula, ρ i is the resource competition conflict factor, C k is the current resource usage of the kth hardware device, R k is the total resource capacity of the kth device, m is the total number of hardware devices involved in executing the task, and β is the control coefficient of resource competition intensity; Introducing the deadlock potential risk coefficient, the calculation expression is as follows: In the formula, δ i represents the deadlock potential risk coefficient, which is used to evaluate the synchronization waiting risk caused by excessive pipeline depth at the i-th time point. γ is the deadlock sensitivity coefficient, which is used to control the impact of depth difference on deadlock risk. Based on pipeline synchronization dependency coefficient λ i , resource competition conflict factor ρ i And the deadlock potential risk factor δ i , generate a comprehensive execution pipeline depth index, the calculation expression is as follows: Where EPDI represents the execution pipeline depth index, and θ is the execution pipeline depth index adjustment parameter, which is used to balance the impact of the depth value on the index.

5. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 3, characterized in that: The logic of performing abnormal analysis on concurrency control information and generating concurrency control index is as follows: In the detection window, we first evaluate the resources occupied by each task during execution, including memory, CPU, and communication bandwidth. The calculation expression is as follows: In the formula, R e is the resource occupancy rate, which indicates the proportion of resources occupied by each task e in its life cycle, U m is the memory usage of task e, T cpu is the time that task e consumes on the CPU, U b is the occupancy rate of task e on the communication bandwidth, T comm is the communication time of task e, C max is the maximum resource capacity of the system, T total is the total execution time of task e; Different tasks need to complete data exchange and collaborative execution synchronously. Synchronization overhead is an important factor in measuring concurrent task scheduling. The synchronization overhead calculation expression is as follows: In the formula, S e is the synchronization overhead, which represents the ratio of the synchronization time required by task e to its total execution time, T syne It is the time that task e waits for other tasks or devices to complete synchronization during execution; Another key factor in concurrency control is resource competition. When system resources are limited, competition will occur when multiple tasks compete for resources at the same time. The resource competition factor is introduced and the calculation expression is as follows: In the formula, C f is the resource competition factor, T avail (s) represents the remaining amount of available resources in the system at time point s, and p is the total number of tasks; Combined with the resource utilization rate R e , synchronization overhead S e and resource competition factor C f And the remaining amount of system available resources T avail (s) and calculate the system concurrent pressure. The calculation expression is as follows: Where P sys is the system concurrency pressure, T limit (s) is the maximum allowed execution time of each task in the system at time point s, which is related to the system load and scheduling strategy; The concurrency control index is generated through comprehensive calculation of system concurrency pressure and resource competition factors. The calculation expression is as follows: Where CCI stands for Concurrency Control Index.

6. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 3, characterized in that: The execution pipeline depth index EPDI and the concurrency control index CCI are input into the pre-learned machine model, and the abnormal scheduling index AS is generated through the machine model. The formula is as follows: Wherein, V1 and V2 are the preset proportional coefficients of the execution pipeline depth index EPDI and the concurrency control index CCI, respectively, and both V1 and V2 are greater than 0.

7. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 6, characterized in that: The generated abnormal scheduling index is compared with the preset abnormal scheduling index reference threshold, and the analysis results are as follows: If the abnormal scheduling index is greater than or equal to a preset abnormal scheduling index reference threshold, the convolutional neural network task scheduling process is classified as an abnormal scheduling state; If the abnormal scheduling index is less than the preset abnormal scheduling index reference threshold, the convolutional neural network task scheduling process is divided into a normal scheduling state.

8. The method for accelerating convolutional neural network reasoning based on heterogeneous architecture according to claim 1, characterized in that: When an abnormal scheduling state is generated during the convolutional neural network task scheduling process, a number of abnormal scheduling indices generated subsequently are comprehensively analyzed with the pre-set abnormal scheduling index reference threshold to generate a scheduling evaluation index DD, based on the formula: Where, AS r represents an abnormal scheduling index that is greater than or equal to an abnormal scheduling index reference threshold, r represents the number of the abnormal scheduling index that is greater than or equal to the abnormal scheduling index reference threshold, r=1, 2, 3..., h, and h is a positive integer. The generated dispatch evaluation index is compared and analyzed with the preset dispatch evaluation index reference threshold, and the analysis results are as follows: If the scheduling evaluation index is greater than or equal to the preset scheduling evaluation index reference threshold, the convolutional neural network task scheduling process is classified as a non-accidental abnormality; If the scheduling evaluation index is less than the preset scheduling evaluation index reference threshold, the convolutional neural network task scheduling process is classified as an accidental abnormality.

9. A convolutional neural network reasoning acceleration system based on a heterogeneous architecture, used to implement the convolutional neural network reasoning acceleration method based on a heterogeneous architecture as described in any one of claims 1 to 8 above, characterized in that: It includes parameter acquisition module, exception analysis and perception module, scheduling status judgment module, exception type identification module and exception handling module: A parameter acquisition module, during the convolutional neural network task scheduling, obtains various parameter information generated during the convolutional neural network task scheduling process, including task scheduling parameters and parallel processing parameters; The abnormality analysis and perception module performs abnormality analysis on the acquired task scheduling parameters and parallel processing parameters under the detection window, and inputs the task scheduling parameters and parallel processing parameters after abnormality analysis into the pre-learned machine model, generates abnormal scheduling index through the machine model, and performs intelligent perception of the convolutional neural network task scheduling process based on the abnormal scheduling index; The scheduling state judgment module divides the convolutional neural network task scheduling process into abnormal scheduling state or normal scheduling state based on the output of the machine learning model, namely the abnormal scheduling index; The abnormal type identification module continuously obtains the operation information of the convolutional neural network task scheduling process for the abnormal scheduling state, and further divides the abnormal scheduling state into accidental abnormality or non-accidental abnormality; The exception handling module adopts different processing methods for different abnormal scheduling states.

Images