Automatic resource allocation method for bearing neural network

Abstract

The invention relates to the technical field of computers, in particular to an automatic resource allocation method for bearing a neural network, and the method comprises the following steps: S1, analyzing the calculation precision demand of each network layer according to the hierarchical structure and calculation characteristics of the neural network; s2, predicting hardware resources of processing unit computing power and memory bandwidth required by each computing stage; s3, automatically adjusting the resource allocation of the task according to the resource prediction result of the S2; s4, dynamically adjusting a calculation precision mode according to the resource load; and S5, after the task is completed, the allocated hardware resources are released, and a resource use feedback report is generated. According to the method, by intelligently analyzing the calculation precision requirement and the hardware resource condition of the neural network task, dynamically adjusting the calculation precision mode and optimizing resource allocation, the utilization efficiency of hardware resources is remarkably improved, the calculation complexity is reduced, and resource waste and bottleneck problems are effectively avoided.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an automatic resource allocation method for carrying a neural network. Background Art

[0002] With the rapid development of deep learning technology, the application of neural networks in many fields such as image recognition, natural language processing, and speech recognition has made remarkable progress; the computing tasks of neural network models usually require a lot of hardware resources, especially when dealing with complex calculations and large-scale data, the demand for CPU, GPU and memory bandwidth increases sharply; in order to improve the computing efficiency of neural networks and reduce the consumption of hardware resources, researchers have proposed the concept of mixed precision computing, that is, using different precision modes in different computing stages according to the computing requirements of the task; this method can not only effectively reduce the computational complexity, but also reduce the use of hardware resources; however, how to intelligently switch between different computing precision modes to achieve optimal resource utilization and computing efficiency is still a technical problem that needs to be solved urgently.

[0003] At present, although there are certain resource allocation methods for hardware scheduling of neural networks, the existing technologies mainly focus on static resource allocation or dynamic adjustment based on experience, and lack a comprehensive analysis and intelligent prediction mechanism for the computing accuracy requirements of tasks; the existing methods have the following shortcomings in the face of differences in computing intensity in neural network tasks, uneven utilization of hardware resources, and dynamic adjustment of accuracy during task execution: First, it is impossible to accurately analyze the computing characteristics and accuracy requirements of each neural network layer, resulting in inefficient resource allocation; second, the existing resource allocation strategies often do not make flexible adjustments based on the real-time load of the task, resulting in resource waste or bottlenecks; third, the existing resource allocation methods lack dynamic monitoring and feedback mechanisms, and fail to respond in real time to the usage status of hardware resources and the need to adjust computing accuracy. Summary of the invention

[0004] Based on the above objectives, the present invention provides an automatic resource allocation method for carrying a neural network.

[0005] An automated resource allocation method for carrying a neural network comprises the following steps: S1: Analyze the computational accuracy requirements of each network layer based on the hierarchical structure and computational characteristics of the neural network, and select the computational accuracy mode based on the computational intensity of the task; S2: Based on the calculation accuracy requirements determined in S1, the load prediction model is used in combination with the hardware performance to predict the hardware resources of the processing unit computing power and memory bandwidth required for each calculation stage; S3: Automatically adjust the resource allocation of tasks based on the resource prediction results of S2; S4: During task execution, the usage of hardware resources is monitored in real time, and the calculation accuracy mode is dynamically adjusted according to the resource load; S5: After the task is completed, the allocated hardware resources are released and a resource usage feedback report is generated.

[0006] Optionally, the S1 specifically includes: S11: By analyzing the topological structure of the neural network, identify the type of each network layer and its position in the entire network, and determine the function and role of each layer; S12: Evaluate the computing characteristics of each network layer, including computing density, memory access pattern, and data dependency, and obtain the specific computing requirement parameters of each layer; S13: Based on the computing characteristics evaluated in step S12, determine the computing accuracy required for each network layer, and select the corresponding computing accuracy mode to match the computing requirements of the layer. The computing accuracy mode includes the full-precision mode FP32, the half-precision mode FP16, and the low-precision mode INT8; for each layer, the computing accuracy mode is determined by the following formula: ; in, Indicates the selected precision mode. Indicates the computational intensity of the current network layer, and are the preset high computing density and low computing density thresholds respectively; according to the different computing densities, the corresponding computing precision mode is selected: when the computing density is higher than Select full precision mode FP32 when the computational density is between and Select half-precision mode FP16 when the computational intensity is lower than Select low precision mode INT8.

[0007] Optionally, the S2 specifically includes: S21: collects performance data of each processing unit in the system in real time, including clock frequency, core utilization, and memory bandwidth utilization, as basic data for resource demand prediction; S22: Use historical task execution data to train a load prediction model based on a support vector machine algorithm; the load prediction model can learn and identify resource demand patterns for processing unit computing power and memory bandwidth at different computing stages according to the computing accuracy requirements of the task; S23: The calculation accuracy requirement determined in S1 is used as input, combined with the hardware performance data collected in step S21, and input into the load prediction model to output the specific values ​​of the processing unit computing power and memory bandwidth requirements required for each calculation stage.

[0008] Optionally, the S23 specifically includes: S221: Collecting execution data from multiple historical tasks, including the computational accuracy requirement of each task, the computational capability of the processing unit, memory bandwidth usage, and computational load of task execution; S222: extracting features from the collected historical task execution data, selecting features related to resource requirements, including computing accuracy, computing density, and memory bandwidth usage; and normalizing the data; S223: using the normalized data as input, training the load forecasting model through a support vector machine algorithm, the support vector machine maximizes the interval between categories by finding the optimal hyperplane, and uses a kernel function to map the data to a high-dimensional space to handle nonlinear relationships; S224: Optimizing the parameters of the support vector machine model using a particle swarm optimization method; the particle swarm optimization iteratively adjusts the parameters of the support vector machine to minimize the prediction error by simulating the coordinated movement of a particle swarm in a search space.

[0009] Optionally, the S224 specifically includes: S2241: randomly initialize a group of particles in the parameter space, each particle represents a set of support vector machine parameters to be optimized; S2242: For each particle, train a support vector machine model according to the parameter group it represents, and evaluate the prediction error of the model on the validation set as the fitness value of the particle; S2243: According to the particle's own optimal position and the global optimal position, the speed and position of each particle are adjusted to push the particle to move to a more optimal parameter area; S2244: repeating steps S2242 and S2243 until a predetermined stop condition is met, including reaching a maximum number of iterations or a fitness value reaching a preset threshold; S2245: Select the best parameter group found in the particle swarm optimization process to build the final support vector machine load prediction model, which is expressed as: ,in, is the predicted resource demand value, and are the parameters of the support vector machine model, is the input feature vector, is the output of the support vector machine model.

[0010] Optionally, the S3 specifically includes: S31: formulating a resource allocation strategy based on the processing unit computing power and memory bandwidth requirements predicted in S2, including determining the resource allocation ratio, priority, and allocation sequence of each processing unit; S32: Apply a resource allocation algorithm based on linear programming to optimize resource allocation of each processing unit, and solve the optimal resource allocation solution by establishing resource constraints and objective functions to maximize resource utilization; S33: According to the resource allocation scheme optimized in S32, the resource allocation of the neural network task on each processing unit is automatically adjusted.

[0011] Optionally, the S32 specifically includes: S321: According to the computing tasks of the neural network and the limitations of hardware resources, define resource constraints, including the maximum computing power of the processing unit, memory bandwidth and storage resources; set the resource constraints as: , ,in, Indicates The computing power of the processing units, Indicates the maximum computing power of the processing unit. is the total number of processing units; S322: To maximize resource utilization and computing performance, establish an objective function; set the objective function to: ,in, Indicates The resource requirements of each processing unit, Indicates The computing power of each processing unit; S323: Add a memory bandwidth constraint to the objective function to ensure that the memory bandwidth utilization is controlled when allocating resources; set the memory bandwidth constraint condition as: ,in, For the The memory bandwidth requirement of each processing unit, is the total memory bandwidth available to the system; S324: According to the resource constraints and objective function established in S321, S322 and S323, the optimal resource allocation scheme is solved by linear programming; the optimal resource allocation value of each processing unit is obtained through the result of the linear programming optimization solution, and the solution expression is: ,in, It is the optimal resource allocation solution obtained through optimization.

[0012] Optionally, the S4 specifically includes: S41: collect and record the performance parameters of the current load, memory usage, bandwidth utilization and temperature of each processing unit in the system in real time; S42: Analyze the real-time hardware resource data collected in step S41 to determine whether the current resource usage status reaches a preset threshold, and identify potential resource bottlenecks or overload conditions; the judgment formula is: load ratio ,in, is the current utilization rate of hardware resources, is the maximum available value of the hardware resources; if the load ratio is greater than or equal to the set threshold of 0.8, it is considered that the current hardware resources are close to being overloaded and there is a potential resource bottleneck; S43: Based on the resource load analysis result in step S42, the calculation accuracy adjustment algorithm is called to automatically determine whether the calculation accuracy mode of the neural network task needs to be adjusted.

[0013] Optionally, the S43 specifically includes: S431: Based on the load ratio calculated in step S42, set the load ratio to , calculate and adjust the decision parameters , and its calculation formula is: , in, Indicates the current load ratio, and These are the preset high load and low load thresholds respectively; this formula is used to automatically decide whether to adjust the calculation accuracy mode; S432: Adjust the decision parameters calculated in S431 , automatically switch the calculation accuracy mode of neural network tasks; like To reduce the precision mode, the calculation precision of the current network layer is switched from full precision mode to low precision mode; like To improve the precision mode, the calculation precision of the current network layer is switched from low-precision mode to full-precision mode; like To maintain the current precision mode, the existing calculation precision configuration remains unchanged.

[0014] Optionally, the S5 specifically includes: S51: After the neural network task is completed, all hardware resources allocated to the task, including CPU cores, GPU units, memory bandwidth and storage resources, are marked and prepared for release; S52: The resource scheduling controller sequentially performs resource recovery operations, specifically including: CPU resources are released, the CPU core assigned to the task is terminated, and it is restored to an idle state; GPU resource release, shut down the GPU unit assigned to the task, release GPU memory bandwidth, and mark GPU resources as reallocatable; Memory bandwidth is released, the memory bandwidth occupation by tasks is relieved, and the memory bandwidth is restored to normal usage; Storage resources are released, storage resources allocated to the task are uninstalled, and the cache is cleared; S53: After the resource release is completed, the resource usage data during the task execution is collected, including the usage time of each processing unit, resource occupancy rate, memory bandwidth utilization and energy consumption data; S54: Organize the resource usage data collected in S54 and generate a resource usage feedback report, including resource allocation, resource usage efficiency, energy consumption statistics, and optimization suggestions.

[0015] Beneficial effects of the present invention: The present invention, by introducing an automated resource allocation method based on calculation accuracy requirement analysis and hardware performance prediction, can intelligently provide efficient resource scheduling and allocation for neural network tasks; by dynamically adjusting the calculation accuracy mode, the system can not only reduce the calculation complexity, but also optimize the utilization of hardware resources, and realize efficient use of resources during task execution; compared with the prior art, this scheme establishes an effective connection between calculation accuracy and hardware resource management, and can switch between multiple accuracy modes in real time according to the actual requirements of the task and the calculation load, thereby achieving the effect of reducing unnecessary resource waste and optimizing calculation efficiency.

[0016] The present invention ensures efficient use of system resources and avoids overload or resource bottlenecks by continuously monitoring the usage of hardware resources and dynamically adjusting the calculation accuracy according to the load analysis results. After the task is completed, the system automatically releases the allocated hardware resources and generates a resource usage feedback report to provide data support for subsequent resource allocation strategy optimization. This intelligent resource allocation and management method greatly improves the execution efficiency of neural network tasks and reduces system energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of an automatic resource allocation method for carrying a neural network according to an embodiment of the present invention; Figure 2 The figure is a schematic diagram of a resource allocation process for automatically adjusting a task according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0020] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0021] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0022] like Figure 1-Figure 2 As shown, an automatic resource allocation method for carrying a neural network includes the following steps: S1: Analyze the computational accuracy requirements of each network layer based on the hierarchical structure and computational characteristics of the neural network, and select the computational accuracy mode based on the computational intensity of the task; S2: Based on the calculation accuracy requirements determined in S1, the load prediction model is used in combination with the hardware performance to predict the hardware resources of the processing unit computing power and memory bandwidth required for each calculation stage; S3: Automatically adjust the resource allocation of tasks based on the resource prediction results of S2. If the system resources are insufficient, the calculation accuracy of the task is reduced; if the resources are sufficient, the calculation accuracy is improved. S4: During task execution, the usage of hardware resources is monitored in real time, and the calculation accuracy mode is dynamically adjusted according to the resource load; S5: After the task is completed, the allocated hardware resources are released and a resource usage feedback report is generated for reference when optimizing resource allocation for subsequent tasks.

[0023] S1 specifically includes: S11: By analyzing the topological structure of the neural network, identify the type of each network layer (such as convolutional layer, fully connected layer, pooling layer, etc.) and its position in the entire network, and determine the function and role of each layer; S12: Evaluate the computing characteristics of each network layer, including computing density, memory access pattern, and data dependency, and obtain the specific computing requirement parameters of each layer; S13: Based on the computing characteristics evaluated in step S12, determine the computing accuracy required for each network layer, and select the corresponding computing accuracy mode to match the computing requirements of the layer. The computing accuracy modes include full-precision mode FP32, half-precision mode FP16, and low-precision mode INT8. For each layer, the computing accuracy mode is determined by the following formula: ; in, Indicates the selected precision mode. Indicates the computational intensity of the current network layer, and are the preset high computing density and low computing density thresholds respectively; according to the different computing densities, the corresponding computing precision mode is selected: when the computing density is higher than Select full precision mode FP32 when the computational density is between and Select half-precision mode FP16 when the computational intensity is lower than Select the low-precision mode INT8; through the above steps S11 to S13, the method dynamically selects the appropriate calculation accuracy mode according to the calculation intensity of each network layer, and combines the specific threshold division strategy to ensure that the selection of the accuracy mode meets the calculation characteristic requirements of each layer; this method can optimize resource allocation, reduce energy consumption, and improve system processing capabilities while ensuring calculation efficiency.

[0024] S2 specifically includes: S21: Real-time collection of performance data of each processing unit (including CPU, GPU, etc.) in the system, including clock frequency, core utilization, and memory bandwidth utilization, as basic data for resource demand prediction; S22: Use historical task execution data to train a load prediction model based on a support vector machine algorithm; the load prediction model can learn and identify resource demand patterns for processing unit computing power and memory bandwidth at different computing stages according to the computing accuracy requirements of the task; S23: The computing accuracy requirement determined in S1 is used as input, combined with the hardware performance data collected in step S21, and input into the load prediction model, and the specific values ​​of the processing unit computing power and memory bandwidth requirements required for each computing stage are output as a reference for subsequent resource allocation; through the above steps S21 to S23, the method can accurately collect and analyze hardware performance data, and use the load prediction model trained based on the support vector machine algorithm to accurately predict the processing unit computing power and memory bandwidth required for each computing stage. This process ensures the accuracy and reliability of resource demand prediction, and provides a solid technical foundation for subsequent dynamic resource allocation.

[0025] S23 specifically includes: S221: Collect execution data from multiple historical tasks, including the computational accuracy requirements of each task, the computing power of the processing unit (such as CPU / GPU core frequency, core utilization, etc.), memory bandwidth usage, and the computational load of task execution (such as the computational density of each network layer, data transmission volume, etc.); S222: extract features from the collected historical task execution data and select features related to resource requirements, including computing accuracy, computing density, and memory bandwidth usage; normalize the data to ensure that data of different dimensions have a balanced impact on the training model and avoid certain features from having too much impact on the model; S223: Use the normalized data as input and train the load forecasting model through the support vector machine algorithm. The support vector machine maximizes the interval between categories by finding the optimal hyperplane and uses the kernel function to map the data to a high-dimensional space to handle nonlinear relationships. Its optimization goal is: ,in, is the normal vector of the hyperplane, is the slack variable, is the penalty parameter, is the number of samples; this optimization objective is used to find the optimal classification hyperplane so that the model can accurately predict the hardware resources required for each computing stage; S224: Particle Swarm Optimization (PSO) is used to optimize the parameters of the support vector machine model. PSO simulates the coordinated movement of a swarm of particles in the search space and iteratively adjusts the parameters of the support vector machine (such as the penalty parameter and kernel function parameters) to minimize the prediction error.

[0026] The particle swarm optimization steps in S224 specifically include: S2241: randomly initialize a group of particles in the parameter space, each particle represents a set of support vector machine parameters to be optimized; S2242: For each particle, train a support vector machine model according to the parameter group it represents, and evaluate the prediction error of the model on the validation set as the fitness value of the particle; S2243: According to the particle's own optimal position and the global optimal position, the speed and position of each particle are adjusted to push the particle to move to a more optimal parameter area; S2244: repeating steps S2242 and S2243 until a predetermined stop condition is met, including reaching a maximum number of iterations or a fitness value reaching a preset threshold; S2245: Select the best parameter group found in the particle swarm optimization process to build the final support vector machine load prediction model, which is expressed as: ,in, is the predicted resource requirement value (such as computing power and memory bandwidth), and are the parameters of the support vector machine model, is the input feature vector, is the output of the support vector machine model; the model predicts the processing unit computing power and memory bandwidth required for each computing stage by inputting the computing accuracy requirements and other key features of the task; through the above steps, it is possible to systematically collect and process historical task data, use the support vector machine algorithm and combine the particle swarm optimization method to accurately optimize the model parameters, and thus build an efficient and accurate load prediction model; the model can accurately predict the processing unit computing power and memory bandwidth required for each computing stage of the neural network, provide a reliable reference for subsequent dynamic resource allocation, and significantly improve the intelligence level of resource allocation and the overall performance of the system.

[0027] S3 specifically includes: S31: formulating a resource allocation strategy based on the processing unit computing power and memory bandwidth requirements predicted in S2, including determining the resource allocation ratio, priority, and allocation sequence of each processing unit; S32: Apply the resource allocation algorithm based on linear programming (LP) to optimize the resource allocation of each processing unit. By establishing resource constraints and objective functions, the optimal resource allocation solution is solved to ensure that the resource requirements of each computing stage are met and the resource utilization rate is maximized. S33: According to the resource allocation plan optimized in S32, the resource allocation of the neural network task on each processing unit is automatically adjusted, including dynamically adjusting the number of CPU and GPU cores, memory bandwidth allocation, and storage resource allocation to ensure that the task can run efficiently with the allocated resources.

[0028] S32 specifically includes: S321: According to the computing tasks of the neural network and the limitations of hardware resources, define resource constraints, including the maximum computing power of the processing unit, memory bandwidth and storage resources; set the resource constraints as: , ,in, Indicates The computing power of the processing units, Indicates the maximum computing power of the processing unit. is the total number of processing units; S322: To maximize resource utilization and computing performance, establish an objective function; the objective function is designed to minimize resource waste and ensure that the computing requirements of the neural network task are met; set the objective function to: ,in, Indicates The resource requirements of each processing unit, Indicates The computing power of each processing unit, the objective function aims to make the actual resource allocation as close as possible to the required resources to maximize the efficiency of resource utilization; S323: Add a memory bandwidth constraint to the objective function to ensure that the memory bandwidth utilization is controlled when allocating resources; set the memory bandwidth constraint condition as: ,in, For the The memory bandwidth requirement of each processing unit, is the total memory bandwidth available to the system; S324: According to the resource constraints and objective function established in S321, S322 and S323, the optimal resource allocation scheme is solved by linear programming; the optimal resource allocation value of each processing unit is obtained through the result of the linear programming optimization solution, and the solution expression is: ,in, is the optimal resource allocation solution obtained through optimization; through the above steps S321 to S324, the resource constraints and objective functions constructed by the linear programming algorithm effectively optimize the resource allocation; the constraints ensure that the resource allocation of each processing unit does not exceed its hardware capabilities, while the objective function maximizes resource utilization and reduces resource waste in the calculation process; through the optimized resource allocation solution, the system can allocate the computing power and memory bandwidth of the processing unit in a more efficient manner, thereby improving the execution efficiency of the neural network computing tasks, ensuring that the computing requirements are met and maximizing resource utilization.

[0029] S4 specifically includes: S41: collects and records the performance parameters of the current load, memory usage, bandwidth utilization, and temperature of each processing unit (including CPU, GPU, etc.) in the system in real time to ensure comprehensive monitoring of resource usage; S42: Analyze the real-time hardware resource data collected in step S41 to determine whether the current resource usage status reaches a preset threshold, and identify potential resource bottlenecks or overload conditions; the judgment formula is: load ratio ,in, is the current utilization rate of hardware resources, is the maximum available value of the hardware resources (such as the maximum CPU frequency, the maximum GPU computing power, the maximum memory bandwidth, etc.); if the load ratio is greater than or equal to the set threshold of 0.8, it is considered that the current hardware resources are close to being overloaded and there is a potential resource bottleneck; it also analyzes the correlation between different resources, such as the usage relationship between the CPU and memory. If it is detected that any resource exceeds the preset threshold, it is considered that there is a potential resource bottleneck or overload; S43: Based on the resource load analysis result in step S42, the calculation accuracy adjustment algorithm is called to automatically determine whether the calculation accuracy mode of the neural network task needs to be adjusted; if it is detected that the resource load is too high, the calculation accuracy mode is reduced; if the resource load is low, the calculation accuracy mode is increased; through the above steps S41 to S43, the method can monitor and analyze the usage of hardware resources in real time during the execution of the neural network task, and dynamically adjust the calculation accuracy mode according to the changes in resource load; this dynamic adaptation mechanism ensures that the calculation accuracy can be reduced to maintain the stable operation of the system when resources are tight, and the calculation accuracy can be increased to optimize the model performance when resources are abundant.

[0030] S43 specifically includes: S431: Based on the load ratio calculated in step S42, set the load ratio to , calculate and adjust the decision parameters , and its calculation formula is: , in, Indicates the current load ratio, and These are the preset high load and low load thresholds respectively; this formula is used to automatically decide whether to adjust the calculation accuracy mode; S432: Adjust the decision parameters calculated in S431 , automatically switch the calculation accuracy mode of neural network tasks; like To reduce the precision mode, the calculation precision of the current network layer is switched from full precision mode to low precision mode; like To improve the precision mode, the calculation precision of the current network layer is switched from low-precision mode to full-precision mode; like To maintain the current accuracy mode, the existing calculation accuracy configuration is maintained unchanged; through the above steps, it is automatically determined whether the calculation accuracy mode of the neural network task needs to be adjusted. This method effectively improves the intelligence level of resource allocation and optimizes computing efficiency.

[0031] S5 specifically includes: S51: After the neural network task is completed, all hardware resources allocated to the task, including CPU cores, GPU units, memory bandwidth and storage resources, are marked and prepared for release; S52: The resource scheduling controller performs resource recovery operations in sequence, including: CPU resources are released, the CPU core assigned to the task is terminated, and it is restored to an idle state; GPU resource release, shut down the GPU unit assigned to the task, release GPU memory bandwidth, and mark GPU resources as reallocatable; Memory bandwidth is released, the memory bandwidth occupation by tasks is relieved, and the memory bandwidth is restored to normal usage; Storage resources are released, storage resources allocated to the task are uninstalled, and the cache is cleared; S53: After the resource release is completed, the resource usage data during the task execution is collected, including the usage time of each processing unit, resource occupancy rate, memory bandwidth utilization and energy consumption data, which are used as basic information for resource usage feedback; S54: Organize the resource usage data collected in S54 and generate a resource usage feedback report, including resource allocation status, resource usage efficiency, energy consumption statistics and optimization suggestions; through the above steps S51 to S54, after the neural network task is completed, the allocated hardware resources can be systematically released and a resource usage feedback report can be accurately generated. The resource release mechanism ensures the timely recovery and reuse of system resources, avoids resource idleness or conflict, and the comprehensive collection of resource usage data and the generation of feedback reports provide a detailed basis for the subsequent optimization of resource allocation strategies.

[0032] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0033] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for automatic resource allocation for carrying a neural network, characterized in that: The following steps are involved: S1: Analyze the computational accuracy requirements of each network layer based on the hierarchical structure and computational characteristics of the neural network, and select the computational accuracy mode based on the computational intensity of the task; S2: Based on the calculation accuracy requirements determined in S1, the load prediction model is used in combination with the hardware performance to predict the hardware resources of the processing unit computing power and memory bandwidth required for each calculation stage; S3: Automatically adjust the resource allocation of tasks based on the resource prediction results of S2; S4: During task execution, the usage of hardware resources is monitored in real time, and the calculation accuracy mode is dynamically adjusted according to the resource load; S5: After the task is completed, the allocated hardware resources are released and a resource usage feedback report is generated.

2. The method for automatic resource allocation for carrying a neural network according to claim 1, characterized in that: The S1 specifically includes: S11: By analyzing the topological structure of the neural network, identify the type of each network layer and its position in the entire network, and determine the function and role of each layer; S12: Evaluate the computing characteristics of each network layer, including computing density, memory access pattern, and data dependency, and obtain the specific computing requirement parameters of each layer; S13: Based on the computing characteristics evaluated in step S12, determine the computing accuracy required for each network layer, and select the corresponding computing accuracy mode to match the computing requirements of the layer. The computing accuracy mode includes the full-precision mode FP32, the half-precision mode FP16, and the low-precision mode INT8; for each layer, the computing accuracy mode is determined by the following formula: ； in, Indicates the selected precision mode. Indicates the computational intensity of the current network layer, and are the preset high computing density and low computing density thresholds respectively; according to the different computing densities, the corresponding computing precision mode is selected: when the computing density is higher than Select full precision mode FP32 when the computational density is between and Select half-precision mode FP16 when the computational intensity is lower than Select low precision mode INT8.

3. The method for automatic resource allocation for carrying a neural network according to claim 1, characterized in that: The S2 specifically includes: S21: collects performance data of each processing unit in the system in real time, including clock frequency, core utilization, and memory bandwidth utilization, as basic data for resource demand prediction; S22: Use historical task execution data to train a load prediction model based on a support vector machine algorithm; the load prediction model can learn and identify resource demand patterns for processing unit computing power and memory bandwidth at different computing stages according to the computing accuracy requirements of the task; S23: The calculation accuracy requirement determined in S1 is used as input, combined with the hardware performance data collected in step S21, and input into the load prediction model to output the specific values ​​of the processing unit computing power and memory bandwidth requirements required for each calculation stage.

4. The method for automatic resource allocation for carrying a neural network according to claim 3, characterized in that: The S23 specifically includes: S221: Collecting execution data from multiple historical tasks, including the computational accuracy requirement of each task, the computational capability of the processing unit, memory bandwidth usage, and computational load of task execution; S222: extracting features from the collected historical task execution data, selecting features related to resource requirements, including computing accuracy, computing density, and memory bandwidth usage; and normalizing the data; S223: using the normalized data as input, training the load forecasting model through a support vector machine algorithm, the support vector machine maximizes the interval between categories by finding the optimal hyperplane, and uses a kernel function to map the data to a high-dimensional space to handle nonlinear relationships; S224: Optimizing the parameters of the support vector machine model using a particle swarm optimization method; the particle swarm optimization iteratively adjusts the parameters of the support vector machine to minimize the prediction error by simulating the coordinated movement of a particle swarm in a search space.

5. The method for automatic resource allocation for carrying a neural network according to claim 4, characterized in that: The S224 specifically includes: S2241: randomly initialize a group of particles in the parameter space, each particle represents a set of support vector machine parameters to be optimized; S2242: For each particle, train a support vector machine model according to the parameter group it represents, and evaluate the prediction error of the model on the validation set as the fitness value of the particle; S2243: According to the particle's own optimal position and the global optimal position, the speed and position of each particle are adjusted to push the particle to move to a more optimal parameter area; S2244: repeating steps S2242 and S2243 until a predetermined stop condition is met, including reaching a maximum number of iterations or a fitness value reaching a preset threshold; S2245: Select the best parameter group found in the particle swarm optimization process to build the final support vector machine load prediction model, which is expressed as: ,in, is the predicted resource demand value, and are the parameters of the support vector machine model, is the input feature vector, is the output of the support vector machine model.

6. The method for automatic resource allocation for carrying a neural network according to claim 3, characterized in that: The S3 specifically includes: S31: formulating a resource allocation strategy based on the processing unit computing power and memory bandwidth requirements predicted in S2, including determining the resource allocation ratio, priority, and allocation sequence of each processing unit; S32: Apply a resource allocation algorithm based on linear programming to optimize resource allocation of each processing unit, and solve the optimal resource allocation solution by establishing resource constraints and objective functions to maximize resource utilization; S33: According to the resource allocation scheme optimized in S32, the resource allocation of the neural network task on each processing unit is automatically adjusted.

7. The method of automatic resource allocation for carrying a neural network according to claim 6, characterized in that: The S32 specifically includes: S321: According to the computing tasks of the neural network and the limitations of hardware resources, define resource constraints, including the maximum computing power of the processing unit, memory bandwidth and storage resources; set the resource constraints as: , ,in, Indicates The computing power of the processing units, Indicates the maximum computing power of the processing unit. is the total number of processing units; S322: To maximize resource utilization and computing performance, establish an objective function; set the objective function to: ,in, Indicates The resource requirements of each processing unit, Indicates The computing power of each processing unit; S323: Add a memory bandwidth constraint to the objective function to ensure that the memory bandwidth utilization is controlled when allocating resources; set the memory bandwidth constraint condition as: ,in, For the The memory bandwidth requirement of each processing unit, is the total memory bandwidth available to the system; S324: According to the resource constraints and objective function established in S321, S322 and S323, the optimal resource allocation scheme is solved by linear programming; the optimal resource allocation value of each processing unit is obtained through the result of the linear programming optimization solution, and the solution expression is: ,in, It is the optimal resource allocation solution obtained through optimization.

8. The method of automatic resource allocation for carrying a neural network according to claim 1, characterized in that: The S4 specifically includes: S41: collect and record the performance parameters of the current load, memory usage, bandwidth utilization and temperature of each processing unit in the system in real time; S42: Analyze the real-time hardware resource data collected in step S41 to determine whether the current resource usage status reaches a preset threshold, and identify potential resource bottlenecks or overload conditions; the judgment formula is: load ratio ,in, is the current utilization rate of hardware resources, is the maximum available value of the hardware resources; if the load ratio is greater than or equal to the set threshold of 0.8, it is considered that the current hardware resources are close to being overloaded and there is a potential resource bottleneck; S43: Based on the resource load analysis result in step S42, the calculation accuracy adjustment algorithm is called to automatically determine whether the calculation accuracy mode of the neural network task needs to be adjusted.

9. The method for automatic resource allocation for carrying a neural network according to claim 8, characterized in that: The S43 specifically includes: S431: Based on the load ratio calculated in step S42, set the load ratio to , calculate and adjust the decision parameters , and its calculation formula is: ， in, Indicates the current load ratio, and These are the preset high load and low load thresholds respectively; this formula is used to automatically decide whether to adjust the calculation accuracy mode; S432: Adjust the decision parameters calculated in S431 , automatically switch the calculation accuracy mode of neural network tasks; like To reduce the precision mode, the calculation precision of the current network layer is switched from full precision mode to low precision mode; like To improve the precision mode, the calculation precision of the current network layer is switched from low-precision mode to full-precision mode; like To maintain the current precision mode, the existing calculation precision configuration remains unchanged.

10. The method for automatic resource allocation for carrying a neural network according to claim 1, characterized in that: The S5 specifically includes: S51: After the neural network task is completed, all hardware resources allocated to the task, including CPU cores, GPU units, memory bandwidth and storage resources, are marked and prepared for release; S52: The resource scheduling controller sequentially performs resource recovery operations, specifically including: CPU resources are released, the CPU core assigned to the task is terminated, and it is restored to an idle state; GPU resource release, shut down the GPU unit assigned to the task, release GPU memory bandwidth, and mark GPU resources as reallocatable; Memory bandwidth is released, the memory bandwidth occupation by tasks is relieved, and the memory bandwidth is restored to normal usage; Storage resources are released, storage resources allocated to the task are uninstalled, and the cache is cleared; S53: After the resource release is completed, the resource usage data during the task execution is collected, including the usage time of each processing unit, resource occupancy rate, memory bandwidth utilization and energy consumption data; S54: Organize the resource usage data collected in S54 and generate a resource usage feedback report, including resource allocation, resource usage efficiency, energy consumption statistics, and optimization suggestions.

Images