A deep learning-based large model deployment and inference method in a low-resource environment

Abstract

The application discloses a low-resource environment large model deployment reasoning method based on deep learning, comprising the following steps: S1, acquiring hardware physical characteristic parameters and constructing a hardware perception cost model; S2, generating an initial lightweight model by using a neural architecture search algorithm; S3, monitoring the device environment state in real time, calculating the maximum theoretical calculation amount that can be borne, and generating a resource availability vector; S4, calculating semantic entropy values, judging sample complexity and allocating initial calculation resource quotas; S5, introducing an improved CMA-ES algorithm of semantic entropy guided population center initialization, adaptively searching and outputting an execution strategy; S6, performing high or low precision fixed-point operation on key layers and non-key layers respectively; S7, performing decoupling recovery and inverse quantization processing on feature maps, and outputting final results. The application realizes efficient deployment of a large model in a low-resource environment, significantly reduces delay and energy consumption while ensuring reasoning accuracy.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and embedded systems, and in particular to a method for deploying and reasoning large models in low-resource environments based on deep learning. Background Technology

[0002] Deep learning models and adaptive inference algorithms are widely used in fields such as autonomous driving and the Internet of Things due to their powerful feature extraction and dynamic allocation of computing resources. However, in practical applications, low-resource deployment scenarios face challenges such as limited hardware computing power, memory bandwidth bottlenecks, and stringent heat dissipation requirements, which restricts the effectiveness of model deployment.

[0003] Most current methods rely on static architectures and quantization strategies, making it difficult to leverage differences in the semantic complexity of input data, resulting in a lack of flexibility in resource allocation. Some systems rely solely on offline metrics, ignoring dynamic states such as real-time memory availability, CPU / GPU utilization fluctuations, and background task usage, thus limiting the adaptive adjustment capabilities of the strategy.

[0004] Meanwhile, the execution process lacks hardware-aware constraints, making it difficult to finely control energy consumption and latency based on the physical characteristics of the device. This can easily lead to overheating or latency exceeding limits under high loads. In addition, the population initialization of existing evolutionary algorithms is mostly randomized, failing to guide the initial search center based on the semantic entropy value of the samples. This makes it difficult to converge to the optimal solution within a finite number of iterations, and the step size coefficient update does not incorporate resource quota damping adjustment, making it difficult to adapt to changes in sample difficulty and resources. This seriously affects the practical value and stability of the model in real-world scenarios.

[0005] Therefore, how to provide a method for deploying large models in a low-resource environment based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a deep learning-based method for deploying large-scale models in low-resource environments for inference. This invention fully integrates key steps such as hardware physical feature parameter construction, neural architecture search algorithm to generate lightweight networks, an improved CMA-ES algorithm with a semantic entropy-guided population center initialization mechanism, and adaptive execution of policy calls and feature decoupling recovery. It constructs an adaptive inference process with hardware-aware cost modeling, accurate sample complexity assessment, local search of the policy solution space, mixed-precision computation, and detailed feature compensation, achieving efficient utilization of computing resources and a dynamic balance between inference accuracy and resource constraints under dynamic fluctuations. This invention possesses advantages such as lightweight network structure, fast policy search convergence, self-adaptive computational accuracy and bit width, and high resource constraint satisfaction. It can significantly improve inference response speed, computational accuracy, and operational stability under low-resource conditions, thereby effectively solving problems such as high model redundancy, blind policy search, and poor hardware adaptability in existing methods.

[0007] According to an embodiment of the present invention, a method for deploying and inferring large models in low-resource environments based on deep learning includes the following steps: S1. Obtain the hardware physical characteristic parameters of the target low-resource device and construct a hardware-aware cost model; S2. A lightweight attention mechanism network adapted to the physical feature parameters of the hardware is automatically searched and generated using a neural architecture search algorithm. By eliminating redundant connections and low-weight neurons, an initial lightweight model is generated. S3. Monitor the dynamic operating environment status of the equipment in real time, calculate the maximum theoretical amount of computation that the equipment can bear at the current moment, and generate a real-time resource availability vector. S4. Receive the input data to be reasoned, evaluate the feature complexity, calculate the semantic entropy value of the input data, determine whether the current input data belongs to a simple sample, a medium-complex sample or a difficult sample, and allocate the initial computing resource quota. S5. The resource availability vector and the initial computational resource quota are calculated using the improved CMA-ES algorithm. A semantic entropy-guided population center initialization mechanism is introduced to map the initial search center of the population to the corresponding policy interval. Iterative sampling is performed in the policy solution space. The hardware overhead of the candidate policy is calculated using the hardware-aware cost model. The policy is then adaptively searched and output to the adaptive execution policy. S6. According to the adaptive execution strategy, call the corresponding computation branch in the initial lightweight model, perform high-precision floating-point operations on the critical layer, and perform low-precision fixed-point operations on the non-critical layer. Utilize forward inference propagation to output the feature map. S7. Perform feature decoupling and recovery on the feature map to compensate for the loss of detail caused by low-precision calculation or hierarchical truncation. After dequantization, output the final inference result data.

[0008] Optionally, S1 specifically includes: S11. Read the hardware specifications of the target low-resource device's memory, extract the memory bandwidth peak value, read the access timing data of different memory levels, obtain the access latency of different memory levels, read the thermal design power parameters in the processor specifications, determine the processor heat dissipation temperature threshold, and combine the memory bandwidth peak value, the access latency of different memory levels, and the processor heat dissipation temperature threshold into hardware physical characteristic parameters. S12. Preset a linear regression equation with computation time, energy consumption and temperature as independent variables. Substitute the peak memory bandwidth into the denominator coefficient of the computation time in the linear regression equation, substitute the access latency of different memory levels into the numerator coefficient of the computation time in the linear regression equation, and substitute the processor heat dissipation temperature threshold into the temperature constraint coefficient in the linear regression equation to establish a hardware-aware cost model.

[0009] Optionally, S2 specifically includes: S21. Create a neural architecture search algorithm that includes an encoder, decoder and long short-term memory network controller. Preset a search space consisting of attention mechanism layer, convolutional layer and pooling layer. Call the hardware-aware cost model to calculate the computational cost of the network architecture. S22. Initialize the hidden state vector of the Long Short-Term Memory Network Controller, set all elements of the hidden state vector to 0, input the start marker, obtain the position of the largest activation value in the output layer neuron, read the corresponding integer and assign it to the variable N, execute the loop, read the output vector of the first layer of the Long Short-Term Memory Network Controller, and take the index of the maximum value as the operation type of the current layer. S23. Execute the second loop, input the operation type of the first layer into the long short-term memory network controller, obtain the index of the second largest value as the operation type of the second layer, repeat the execution until the Nth loop ends, combine all the indexes obtained from the first to the Nth loop into a network architecture code in order, input the network architecture code into the encoder, and output the computation graph structure. S24. Read the parameter quantity of each layer in the computation graph structure, multiply the parameter quantity of each layer by the access latency of different memory levels to obtain the computation time of each layer, and add them together to obtain the total computation time. Read the number of floating-point operations of each layer in the computation graph structure, multiply the number of floating-point operations by the processor heat dissipation temperature threshold to obtain the energy consumption value of each layer, and add the energy consumption values ​​of all layers together to obtain the total energy consumption value. S25. If the total computation time exceeds the peak memory bandwidth or the total energy consumption exceeds the preset energy consumption limit, the current network architecture encoding is marked as invalid encoding. Otherwise, the neural network model corresponding to the current network architecture encoding is constructed, the training sample data is input to perform forward propagation, the error value between the output result and the label data is calculated, and the negative number is used to obtain the reward value. S26. Input the reward value into the Long Short-Term Memory Network Controller, update the weight matrix inside the Long Short-Term Memory Network Controller using the backpropagation algorithm, repeat the weight matrix update a preset number of times, extract the network architecture code with the largest reward value, and build the basic network model. S27. Read the weight values ​​of each connection between two neurons in the basic network model. If they are less than the preset weight threshold, delete the edge connecting the two neurons. Read the average value of all values ​​in each convolution kernel in the basic network model. If it is less than the preset average threshold, remove the convolution kernel from the network structure. Repeat the weight matrix update for the basic network model after removing the edge and convolution kernel a preset number of times to obtain the initial lightweight model.

[0010] Optionally, constructing the neural network model corresponding to the current network architecture encoding specifically includes: Create an empty list structure to store network layer objects in order. Read the first value in the network architecture encoding sequence and determine if it is equal to 0. If it is equal to 0, preset the convolution kernel size, number of input channels, number of output channels, stride and padding. Instantiate a two-dimensional convolution operation class and append the instantiated object to the end of the list structure. If it is not equal to 0, then check if the value is equal to 1. If it is equal to 1, then instantiate a multi-head self-attention operation class with a preset embedding dimension size, number of attention heads and dropout rate, and append the instantiated object to the end of the list structure. Read the second value in the network architecture encoding sequence, repeat the judgment and append operation, and read sequentially until the object corresponding to the last value in the current network architecture encoding is instantiated and appended. Encapsulate the list structure containing all objects into the overall neural network model.

[0011] Optionally, S3 specifically includes: S31. Call the underlying performance monitoring interface of the target low-resource device operating system, read the current values ​​in the CPU utilization register and GPU utilization register respectively, record the current CPU value as CPU utilization, record the current GPU value as GPU utilization, read the number of free page addresses in memory management, multiply the number of free page addresses by the size of a single page, and calculate the current available memory balance. S32. Traverse the operating system process list, read the thread time slice of all processes except the current inference process, divide the sum of the time slices occupied by all background tasks by the total CPU clock cycles, calculate the proportion of computing resources occupied by the system background tasks, and record it as the utilization rate. S33. Read the core frequency value from the CPU's hardware specifications, multiply the core frequency value by 1 and subtract the CPU utilization value, then multiply by 1 and subtract the occupancy rate to calculate the maximum theoretical amount of computation that the CPU can currently handle. S34. Read the number of stream processors and core frequency values ​​from the GPU's hardware specifications, multiply the number of stream processors by the core frequency value, multiply by 1 and subtract the GPU utilization value to calculate the maximum theoretical amount of computation that the GPU can currently handle. S35. Arrange the current available memory, the maximum theoretical computational load that the CPU can currently handle, and the maximum theoretical computational load that the GPU can currently handle in sequence, and combine them in order to generate a real-time resource availability vector.

[0012] Optionally, S4 specifically includes: S41. Input the received input data to be inferred into the pre-trained Softmax classification layer, obtain the value of each output node, and sum them to obtain the denominator. Divide the value of each output node by the denominator to obtain the predicted probability value of each category. Call the logarithmic function to calculate the logarithmic value with the natural constant e as the base. Multiply the calculated logarithmic value by the predicted probability value of the corresponding category to obtain the entropy component of each category. Add the values ​​of the entropy components of all categories and record the sum as the semantic entropy value of the input data. S42. Compare the semantic entropy value with the preset entropy range threshold. If the semantic entropy value is less than the lower limit of the entropy range threshold, it is determined to be a simple sample, and the initial computing resource quota is set to 1. If it is within the entropy range threshold, it is determined to be a medium-complex sample, and the initial computing resource quota is set to 2. If it is greater than or equal to the upper limit of the entropy range threshold, it is determined to be a difficult sample, and the initial computing resource quota is set to 3.

[0013] Optionally, S5 specifically includes: S51. Read the real-time resource availability vector, create a three-dimensional coordinate system, set the horizontal axis label to the network depth truncation position, set the vertical axis label to the operator calculation precision bit width, and set the vertical axis label to the data flow path. Read the maximum number of layers at the network depth truncation position, the minimum and maximum bit values ​​of the operator calculation precision bit width, and the total number of data flow path numbers. S52. Set the maximum number of layers as the upper bound of the horizontal axis, set the minimum bit value and the maximum bit value as the lower bound and upper bound of the vertical axis, set the total number of numbers as the upper bound of the vertical axis, and set 0 as the lower bound of all coordinate axes to form a closed three-dimensional cube region as the strategy solution space. S53. Read the semantic entropy value, retrieve the numerical range where the semantic entropy value is located in the preset database, read the preset center point coordinate value corresponding to the numerical range, and assign it to the preset variable as the initial mean vector, and initialize the identity matrix of the preset number of dimensions as the initial covariance matrix. S54. Set the initial step size coefficient, call the random number generation function to generate a preset number of random values ​​that follow a standard normal distribution, generate a random value vector, and perform matrix multiplication with the covariance matrix. Multiply the result by the initial step size coefficient and add it to the initial mean vector to obtain the coordinates of an individual in the population. Repeat the operation of generating individual coordinates until the number of individuals reaches the preset number to obtain the initial population. S55. Traverse each individual in the initial population, read the horizontal axis coordinate value of the individual as the layer index that the network needs to retain, read the vertical axis coordinate value as the number of binary bits used when calculating the network layer, read the vertical axis coordinate value as the path number of data transmission between memory and processor, substitute the layer index, number of binary bits and path number into the hardware-aware cost model, and calculate the total computing time and total energy consumption value required for the current individual to run on the current device. S56. Subtract the total computation time of an individual from the peak memory bandwidth, and subtract the total energy consumption of an individual from the processor heat dissipation temperature threshold. If any result is positive, set the fitness value to 0. Otherwise, configure the layer index, number of bits, and path number corresponding to the current individual into the initial lightweight model, input validation sample data, count the number of correctly predicted results in the output results, divide by the total number of validation sample data, obtain the accuracy value, and use it as the fitness value of the current individual. S57. Sort the fitness of all individuals in the initial population from largest to smallest, select the first preset number of individuals and store them in the excellent set, add the horizontal axis coordinates of all individuals in the excellent set and divide by the total number of individuals in the excellent set to obtain the horizontal axis component of the new generation mean vector, use the same method to calculate the vertical axis component and the vertical axis component, and combine them to obtain the vertical axis component and the vertical axis component of the new generation mean vector. S58. Read the three-dimensional coordinate values ​​of each individual in the excellent set, subtract the coordinate components corresponding to the new generation mean vector to obtain the difference vector, perform matrix multiplication on all difference vectors and their corresponding transpose vectors, add the results of all individuals, divide the sum matrix by the total number of individuals in the excellent set to obtain the updated covariance matrix. S59. Read the initial calculation resource quota value and take its reciprocal to obtain the damping coefficient. Read the evolution path vector of the previous generation iteration. Subtract the mean vector of the new generation from the mean vector of the previous generation to obtain the mean difference vector and divide it by the square root matrix of the covariance matrix to obtain the evolution path vector of the current generation. Perform a weighted sum of the evolution path vector of the current generation and the evolution path vector of the previous generation with preset weights to obtain the updated evolution path. S510. Multiply the values ​​of the main diagonal elements of the covariance matrix by the damping coefficient to obtain the updated covariance matrix. Use the ratio of the magnitude of the updated evolution path vector to the preset expected length as the exponent to perform an exponential operation on the current step size coefficient to obtain a new step size coefficient. Use the updated mean vector, the updated covariance matrix and the new step size coefficient to generate the coordinates of the next generation of individuals through normal distribution sampling. S511. Using the next generation population as the current population, repeatedly execute the operations of reading coordinate values, calculating hardware overhead, comparison threshold, setting fitness values, calculating mean vector, calculating covariance matrix, updating step size coefficient, and generating coordinates of the next generation population. When the number of iterations equals the preset maximum number of iterations, read the horizontal, vertical, and vertical components of the mean vector in the last iteration, and use them as the final network depth truncation position, operator calculation precision bit width, and data flow path, respectively, and combine them in order as the output of the adaptive execution strategy for the current input data.

[0014] Optionally, S6 specifically includes: S61. Read the network depth truncation position value in the adaptive execution strategy, obtain all network layers in the initial lightweight model as a complete sequence, and traverse from the starting layer of the complete sequence according to the network depth truncation position value. Mark all network layers with layer index less than or equal to the network depth truncation position value as retained layers, and remove network layers with layer index greater than the network depth truncation position value from the inference process. S62. For each network layer in the reserved layer, read the operator calculation precision bit width value in the adaptive execution strategy and compare it with the preset precision threshold. If it is greater than or equal to the preset precision threshold, mark the current network layer as a critical layer; otherwise, mark the current network layer as a non-critical layer. S63. Traverse all network layers marked as key layers, perform floating-point operations on the target low-resource device, load the weight data of the key layers, perform matrix multiplication operations between the preset weight tensor and the input feature map using the preset floating-point number format, perform activation operations using the exponential and logarithmic functions, and output the high-precision feature map of the key layers. S64. Traverse all network layers marked as non-critical layers, perform fixed-point operations in the target low-resource device, read the data flow path in the adaptive execution strategy, convert the weight data and input data of non-critical layers into fixed-point number format according to the path instructions, perform dot product and accumulation operations using the preset integer format, and output low-precision feature maps of non-critical layers. S65. Following the connection order of the data flow in the initial lightweight model, the output feature map of the previous layer is used as the input feature map of the next layer. Matrix multiplication, dot product and accumulation operations are repeatedly performed until all retained layers are traversed. The output data of the last layer located at the network depth truncation position is obtained, which is the feature map after inference propagation.

[0015] Optionally, S7 specifically includes: using a preset decoupling algorithm based on wavelet transform or dictionary learning to separate the low-bit quantized feature map in the frequency domain or sparse domain, reconstructing the image edge details and texture information lost due to low-precision truncation through high-frequency components, then performing an inverse quantization operation on the reconstructed feature map to accurately map the fixed-point values ​​back to the original floating-point domain, and outputting the final inference result data.

[0016] The beneficial effects of this invention are: First, by collecting the hardware physical characteristic parameters and dynamic operating environment status of the target low-resource device, a hardware-aware cost model and resource availability vector are constructed, which accurately quantifies the device's computing capacity and heat dissipation limitations at the current moment, providing precise data support for the generation of adaptive strategies.

[0017] Secondly, a neural architecture search algorithm is used to eliminate redundant connections and automatically generate an initial lightweight model adapted to hardware limitations, significantly reducing the number of model parameters and computational complexity. During the inference phase, an improved CMA-ES algorithm is employed, introducing a semantic entropy-guided population center initialization mechanism to map the search range to a policy interval adapted to the sample complexity, significantly improving the search efficiency and convergence speed of the optimal policy. This method, while satisfying hardware overhead constraints, achieves global optimization of the network depth truncation position and operator precision bit width, ensuring the optimal balance between inference accuracy and computational efficiency.

[0018] Furthermore, based on an adaptive execution strategy, heterogeneous computations are performed on critical and non-critical layers. Feature decoupling and recovery mechanisms compensate for the loss of detail caused by low-precision computations or hierarchical truncation, effectively solving the problem of decreased inference accuracy in low-resource environments. The system can dynamically adjust the computation path based on the semantic entropy of the input data and the real-time resource status of the device, ensuring the real-time performance and robustness of the inference process.

[0019] In summary, this invention significantly improves the deployment flexibility, operational stability, and inference accuracy of large models in low-resource environments by integrating neural architecture search, improved evolutionary algorithm optimization, and mixed-precision inference. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a deep learning-based method for deploying and inferring large models in low-resource environments, as proposed in this invention. Figure 2 This is a flowchart of the neural architecture search algorithm based on hardware-aware cost model constraints and the initial lightweight model generation process proposed in this invention. Figure 3 This is a flowchart illustrating the optimization and adaptive execution of the improved CMA-ES algorithm strategy based on the semantic entropy-guided population center initialization mechanism proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A deep learning-based method for deploying large models in low-resource environments for inference includes the following steps: S1. Obtain the hardware physical characteristic parameters of the target low-resource device, including peak memory bandwidth, access latency of different memory levels and processor heat dissipation temperature threshold, and construct a hardware-aware cost model. S2. A lightweight attention mechanism network adapted to the physical feature parameters of the hardware is automatically searched and generated using a neural architecture search algorithm. By eliminating redundant connections and low-weight neurons, an initial lightweight model adapted to the hardware limits is generated. S3. Monitor the dynamic operating environment status of the device in real time, including the current available memory, the real-time utilization fluctuation of CPU and GPU, and the computing resources occupied by background tasks of the system. Calculate the maximum theoretical amount of computing that the device can carry at the current moment and generate a real-time resource availability vector. S4. Receive the input data to be reasoned, evaluate the feature complexity, calculate the semantic entropy value of the input data, determine whether the current input data belongs to a simple sample, a medium-complex sample or a difficult sample, and allocate the initial computing resource quota. S5. The resource availability vector and the initial computational resource quota are calculated using the improved CMA-ES algorithm. A semantic entropy-guided population center initialization mechanism is introduced to map the initial search center of the population to the corresponding policy interval. Iterative sampling is performed in the policy solution space to calculate the hardware overhead of candidate policies. The optimal solution that satisfies the current resource constraints and has the highest inference accuracy is adaptively searched, and an adaptive execution policy for the current input data is output. The adaptive execution policy includes the network depth truncation position, operator computational precision bit width, and data flow path. S6. According to the adaptive execution strategy, call the corresponding computation branch in the initial lightweight model, perform high-precision floating-point operations on the critical layer, and perform low-precision fixed-point operations on the non-critical layer. Utilize the heterogeneous computing architecture to complete the forward inference propagation of the initial lightweight model in a low-resource environment, and output the feature map after inference propagation. S7. Perform feature decoupling and recovery on the feature map to compensate for the loss of detail caused by low-precision calculation or hierarchical truncation. After dequantization, output the final inference result data.

[0023] This invention introduces an improved CMA-ES algorithm combined with a semantic entropy-guided mechanism to achieve adaptive inference and dynamic resource allocation for large models in low-resource environments. It acquires device hardware parameters and environmental states in real time to construct a cost model and resource availability vector, calculates the semantic entropy value of the input data and allocates computational resource quotas, uses semantic entropy to guide population center initialization, and iteratively samples and searches within the policy solution space for an adaptive execution strategy that includes depth truncation and precision bit width. This invention can dynamically adjust computational intensity based on sample complexity and device load, significantly reducing computational power and energy consumption in simple scenarios while ensuring accuracy for high-difficulty samples. It effectively solves the memory overflow and overheating problems when deploying large models on low-resource devices, significantly improving edge inference speed and device battery life.

[0024] In this embodiment, S1 specifically includes: S11. Read the hardware specifications of the target low-resource device's memory, extract the memory bandwidth peak value, read the access timing data of different memory levels, obtain the access latency of different memory levels, read the thermal design power parameters in the processor specifications, determine the processor heat dissipation temperature threshold, and combine the memory bandwidth peak value, the access latency of different memory levels, and the processor heat dissipation temperature threshold into hardware physical characteristic parameters. S12. Preset a linear regression equation with computation time, energy consumption and temperature as independent variables. Substitute the peak memory bandwidth into the denominator coefficient of the computation time in the linear regression equation, substitute the access latency of different memory levels into the numerator coefficient of the computation time in the linear regression equation, and substitute the processor heat dissipation temperature threshold into the temperature constraint coefficient in the linear regression equation to establish a hardware-aware cost model.

[0025] In this embodiment, S2 specifically includes: S21. Create a neural architecture search algorithm that includes an encoder, decoder and long short-term memory network controller. Preset a search space consisting of attention mechanism layer, convolutional layer and pooling layer. Call the hardware-aware cost model to calculate the computational cost of the network architecture. S22. Initialize the hidden state vector of the Long Short-Term Memory Network Controller, set all elements of the hidden state vector to 0, input the start marker, obtain the position of the largest activation value in the output layer neuron, read the corresponding integer and assign it to the variable N, execute the loop, read the output vector of the first layer of the Long Short-Term Memory Network Controller, and take the index of the maximum value as the operation type of the current layer. S23. Execute the second loop, input the operation type of the first layer into the long short-term memory network controller, obtain the index of the second largest value as the operation type of the second layer, repeat the execution until the Nth loop ends, combine all the indexes obtained from the first to the Nth loop into a network architecture code in order, input the network architecture code into the encoder, and output a computation graph structure containing N neural networks. S24. Read the parameter quantity of each layer in the computation graph structure, multiply the parameter quantity of each layer by the access latency of different memory levels to obtain the computation time of each layer, and add them together to obtain the total computation time. Read the number of floating-point operations of each layer in the computation graph structure, multiply the number of floating-point operations by the processor heat dissipation temperature threshold to obtain the energy consumption value of each layer, and add the energy consumption values ​​of all layers together to obtain the total energy consumption value. S25. If the total computation time exceeds the peak memory bandwidth or the total energy consumption exceeds the preset energy consumption limit, the current network architecture encoding is marked as invalid encoding. Otherwise, the neural network model corresponding to the current network architecture encoding is constructed, the training sample data is input to perform forward propagation, the error value between the output result and the label data is calculated, and the negative number is used to obtain the reward value. S26. Input the reward value into the Long Short-Term Memory Network Controller, update the weight matrix inside the Long Short-Term Memory Network Controller using the backpropagation algorithm, repeat the weight matrix update a preset number of times, extract the network architecture code with the largest reward value, and build the basic network model. S27. Read the weight values ​​of each connection between two neurons in the basic network model. If they are less than the preset weight threshold, delete the edge connecting the two neurons. Read the average value of all values ​​in each convolution kernel in the basic network model. If it is less than the preset average threshold, remove the convolution kernel from the network structure. Repeat the weight matrix update for the basic network model after removing the edge and convolution kernel a preset number of times to obtain the initial lightweight model.

[0026] In this embodiment, constructing the neural network model corresponding to the current network architecture encoding specifically includes: Create an empty list structure to store network layer objects in order. Read the first value in the network architecture encoding sequence and determine if it is equal to 0. If it is equal to 0, preset the convolution kernel size, number of input channels, number of output channels, stride and padding. Instantiate a two-dimensional convolution operation class and append the instantiated object to the end of the list structure. If it is not equal to 0, then check if the value is equal to 1. If it is equal to 1, then instantiate a multi-head self-attention operation class with a preset embedding dimension size, number of attention heads and dropout rate, and append the instantiated object to the end of the list structure. Read the second value in the network architecture encoding sequence, repeat the judgment and append operation, and read sequentially until the object corresponding to the last value in the current network architecture encoding is instantiated and appended. Encapsulate the list structure containing all objects into the overall neural network model.

[0027] In this embodiment, S3 specifically includes: S31. Call the underlying performance monitoring interface of the target low-resource device operating system, read the current values ​​in the CPU utilization register and GPU utilization register respectively, record the current CPU value as CPU utilization, record the current GPU value as GPU utilization, read the number of free page addresses in memory management, multiply the number of free page addresses by the size of a single page, and calculate the current available memory balance. S32. Traverse the operating system process list, read the thread time slice of all processes except the current inference process, divide the sum of the time slices occupied by all background tasks by the total CPU clock cycles, calculate the proportion of computing resources occupied by the system background tasks, and record it as the utilization rate. S33. Read the core frequency value from the CPU's hardware specifications, multiply the core frequency value by 1 and subtract the CPU utilization value, then multiply by 1 and subtract the occupancy rate to calculate the maximum theoretical amount of computation that the CPU can currently handle. S34. Read the number of stream processors and core frequency values ​​from the GPU's hardware specifications, multiply the number of stream processors by the core frequency value, multiply by 1 and subtract the GPU utilization value to calculate the maximum theoretical amount of computation that the GPU can currently handle. S35. Arrange the current available memory, the maximum theoretical computational load that the CPU can currently handle, and the maximum theoretical computational load that the GPU can currently handle in sequence, and combine them in order to generate a real-time resource availability vector.

[0028] In this embodiment, S4 specifically includes: S41. Input the received input data to be inferred into the pre-trained Softmax classification layer, obtain the value of each output node, and sum them to obtain the denominator. Divide the value of each output node by the denominator to obtain the predicted probability value of each category. Call the logarithmic function to calculate the logarithmic value with the natural constant e as the base. Multiply the calculated logarithmic value by the predicted probability value of the corresponding category to obtain the entropy component of each category. Add the values ​​of the entropy components of all categories and record the sum as the semantic entropy value of the input data. S42. Compare the semantic entropy value with the preset entropy range threshold. If the semantic entropy value is less than the lower limit of the entropy range threshold, it is determined to be a simple sample, and the initial computing resource quota is set to 1. If it is within the entropy range threshold, it is determined to be a medium-complex sample, and the initial computing resource quota is set to 2. If it is greater than or equal to the upper limit of the entropy range threshold, it is determined to be a difficult sample, and the initial computing resource quota is set to 3.

[0029] In this embodiment, S5 specifically includes: S51. Read the real-time resource availability vector, create a three-dimensional coordinate system, set the horizontal axis label to the network depth truncation position, set the vertical axis label to the operator calculation precision bit width, and set the vertical axis label to the data flow path. Read the maximum number of layers at the network depth truncation position, the minimum and maximum bit values ​​of the operator calculation precision bit width, and the total number of data flow path numbers. S52. Set the maximum number of layers as the upper bound of the horizontal axis, set the minimum bit value and the maximum bit value as the lower bound and upper bound of the vertical axis, set the total number of numbers as the upper bound of the vertical axis, and set 0 as the lower bound of all coordinate axes to form a closed three-dimensional cube region as the strategy solution space. S53. Read the semantic entropy value, retrieve the numerical range where the semantic entropy value is located in the preset database, read the preset center point coordinate value corresponding to the numerical range, and assign it to the preset variable as the initial mean vector, and initialize the identity matrix of the preset number of dimensions as the initial covariance matrix. S54. Set the initial step size coefficient, call the random number generation function to generate a preset number of random values ​​that follow a standard normal distribution, generate a random value vector, and perform matrix multiplication with the covariance matrix. Multiply the result by the initial step size coefficient and add it to the initial mean vector to obtain the coordinates of an individual in the population. Repeat the operation of generating individual coordinates until the number of individuals reaches the preset number to obtain the initial population. S55. Traverse each individual in the initial population, read the horizontal axis coordinate value of the individual as the layer index that the network needs to retain, read the vertical axis coordinate value as the number of binary bits used when calculating the network layer, read the vertical axis coordinate value as the path number of data transmission between memory and processor, substitute the layer index, number of binary bits and path number into the hardware-aware cost model, and calculate the total computing time and total energy consumption value required for the current individual to run on the current device. S56. Subtract the total computation time of an individual from the peak memory bandwidth, and subtract the total energy consumption of an individual from the processor heat dissipation temperature threshold. If any result is positive, set the fitness value to 0. Otherwise, configure the layer index, number of bits, and path number corresponding to the current individual into the initial lightweight model, input validation sample data, count the number of correctly predicted results in the output results, divide by the total number of validation sample data, obtain the accuracy value, and use it as the fitness value of the current individual. S57. Sort the fitness of all individuals in the initial population from largest to smallest, select the first preset number of individuals and store them in the excellent set, add the horizontal axis coordinates of all individuals in the excellent set and divide by the total number of individuals in the excellent set to obtain the horizontal axis component of the new generation mean vector, use the same method to calculate the vertical axis component and the vertical axis component, and combine them to obtain the vertical axis component and the vertical axis component of the new generation mean vector. S58. Read the three-dimensional coordinate values ​​of each individual in the excellent set, subtract the coordinate components corresponding to the new generation mean vector to obtain the difference vector, perform matrix multiplication on all difference vectors and their corresponding transpose vectors, add the results of all individuals, divide the sum matrix by the total number of individuals in the excellent set to obtain the updated covariance matrix. S59. Read the initial calculation resource quota value and take its reciprocal to obtain the damping coefficient. Read the evolution path vector of the previous generation iteration. Subtract the mean vector of the new generation from the mean vector of the previous generation to obtain the mean difference vector and divide it by the square root matrix of the covariance matrix to obtain the evolution path vector of the current generation. Perform a weighted sum of the evolution path vector of the current generation and the evolution path vector of the previous generation with preset weights to obtain the updated evolution path. S510. Multiply the values ​​of the main diagonal elements of the covariance matrix by the damping coefficient to obtain the updated covariance matrix. Use the ratio of the magnitude of the updated evolution path vector to the preset expected length as the exponent to perform an exponential operation on the current step size coefficient to obtain a new step size coefficient. Use the updated mean vector, the updated covariance matrix and the new step size coefficient to generate the coordinates of the next generation of individuals through normal distribution sampling. S511. Using the next generation population as the current population, repeatedly execute the operations of reading coordinate values, calculating hardware overhead, comparison threshold, setting fitness values, calculating mean vector, calculating covariance matrix, updating step size coefficient, and generating coordinates of the next generation population. When the number of iterations equals the preset maximum number of iterations, read the horizontal, vertical, and vertical components of the mean vector in the last iteration, and use them as the final network depth truncation position, operator calculation precision bit width, and data flow path, respectively, and combine them in order as the output of the adaptive execution strategy for the current input data.

[0030] This invention constructs a three-dimensional policy solution space and introduces a semantic entropy-guided CMA-ES algorithm to achieve adaptive optimization of inference strategies for large models. The solution space boundary is set based on resource constraints and sample complexity. The population center is initialized using semantic entropy mapping. By iteratively calculating the hardware overhead and predicted fitness of individual population members, the mean vector, covariance matrix, and evolutionary path are dynamically updated until convergence to the optimal solution. This invention can accurately match network depth, computational bit width, and data flow path, maximizing inference accuracy while meeting device computation time and heat dissipation thresholds. It effectively solves the problem of blind policy search in resource-constrained environments and significantly improves the energy efficiency and operational stability of large model deployments on the edge.

[0031] In this embodiment, S6 specifically includes: S61. Read the network depth truncation position value in the adaptive execution strategy, obtain all network layers in the initial lightweight model as a complete sequence, and traverse from the starting layer of the complete sequence according to the network depth truncation position value. Mark all network layers with layer index less than or equal to the network depth truncation position value as retained layers, and remove network layers with layer index greater than the network depth truncation position value from the inference process. S62. For each network layer in the reserved layer, read the operator calculation precision bit width value in the adaptive execution strategy and compare it with the preset precision threshold. If it is greater than or equal to the preset precision threshold, mark the current network layer as a critical layer; otherwise, mark the current network layer as a non-critical layer. S63. Traverse all network layers marked as key layers, perform floating-point operations on the target low-resource device, load the weight data of the key layers, perform matrix multiplication operations between the preset weight tensor and the input feature map using the preset floating-point number format, perform activation operations using the exponential and logarithmic functions, and output the high-precision feature map of the key layers. S64. Traverse all network layers marked as non-critical layers, perform fixed-point operations in the target low-resource device, read the data flow path in the adaptive execution strategy, convert the weight data and input data of non-critical layers into fixed-point number format according to the path instructions, perform dot product and accumulation operations using the preset integer format, and output low-precision feature maps of non-critical layers. S65. Following the connection order of the data flow in the initial lightweight model, the output feature map of the previous layer is used as the input feature map of the next layer. Matrix multiplication, dot product and accumulation operations are repeatedly performed until all retained layers are traversed. The output data of the last layer located at the network depth truncation position is obtained, which is the feature map after inference propagation.

[0032] In this embodiment, S7 specifically includes: using a preset decoupling algorithm based on wavelet transform or dictionary learning to separate the low-bit quantized feature map in the frequency domain or sparse domain, reconstructing the image edge details and texture information lost due to low-precision truncation through high-frequency components, then performing an inverse quantization operation on the reconstructed feature map to accurately map the fixed-point values ​​back to the original floating-point domain, and outputting the final inference result data.

[0033] Example 1: To verify the application value of this invention in the field of low-resource deployment of large deep learning models, it was applied to a new generation of portable satellite communication ground terminals. This device is designed for scenarios such as emergency rescue in the field. Its core processor is a low-power embedded AI chip, with memory limited to 4GB, and it needs to process massive amounts of radio frequency signal data from the transmitting and receiving sub-mode components in real time. Existing cloud-based collaborative modes experience backhaul latency exceeding 5 seconds in field environments and are prone to interruptions; while directly deploying a large 7B-parameter model locally leads to memory overflow, low inference frame rates, and severe overheating, making it difficult to meet the requirements of low latency and high reliability.

[0034] During deployment, this invention utilizes a hardware-aware neural architecture search algorithm and an improved CMA-ES algorithm guided by semantic entropy to achieve efficient adaptive inference. First, CPU instruction set, NPU computing power, and heat dissipation thresholds are collected to construct a hardware-aware cost model and define resource boundaries. Then, neural architecture search is used to prune the multi-head attention mechanism of the large model, eliminating redundant connections and generating a lightweight architecture suitable for the edge. During the inference phase, the system calculates the semantic entropy of the input RF signal in real time. When encountering complex high-entropy scenarios such as strong interference, the improved CMA-ES algorithm is triggered, guiding population initialization through semantic entropy to quickly locate high-precision strategies and dynamically activate deep attention units; when the channel is good, the computational depth is automatically reduced and low-precision quantization is used. This mechanism enables the large model to dynamically adjust resources according to task difficulty, maximizing utilization while ensuring performance. Table 1 below shows the performance comparison data of the method of this invention with traditional quantization methods and standard pruning methods in various complex communication scenarios: Table 1. Performance Comparison Data Between the Invention and Traditional Methods in Large Model Edge Deployment Scenarios

[0035] Based on the comparative data shown in Table 1, it can be seen that the large model deployment inference method proposed in this invention exhibits significant performance advantages over traditional methods in complex communication environments, especially in terms of comprehensive improvement in key indicators such as inference accuracy, response speed, end-side throughput, and resource consumption control.

[0036] Regarding inference accuracy, this invention maintains a high level of over 94% in all three typical scenarios, far exceeding the average level of approximately 70% for traditional quantization methods and approximately 80% for standard pruning methods. For example, in the scenario of "weak signal covert communication," traditional quantization methods are limited by precision loss, achieving an accuracy of only 69.8%, while this invention, through an adaptive strategy guided by semantic entropy, achieves an accuracy of 96.5%, greatly improving the ability to resolve weak signals. In terms of response timeliness, this invention, by dynamically adjusting the computation depth, significantly reduces the average inference latency from approximately 1900 milliseconds in traditional methods to approximately 150 milliseconds, an improvement of nearly 13 times, ensuring the real-time control capability of the communication system.

[0037] In addition, in terms of resource control, the average power consumption of this invention is reduced to about 10 watts, memory usage is halved, and no overheating protection shutdown occurred during testing. It effectively solves the problems of high power consumption and heat dissipation bottleneck of end-side devices and has broad prospects for promotion and application.

[0038] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for deploying and inferring large models in low-resource environments based on deep learning, characterized in that, Includes the following steps: S1. Obtain the hardware physical characteristic parameters of the target low-resource device and construct a hardware-aware cost model; S2. A lightweight attention mechanism network adapted to the physical feature parameters of the hardware is automatically searched and generated using a neural architecture search algorithm. By eliminating redundant connections and low-weight neurons, an initial lightweight model is generated. S3. Monitor the dynamic operating environment status of the equipment in real time, calculate the maximum theoretical amount of computation that the equipment can bear at the current moment, and generate a real-time resource availability vector. S4. Receive the input data to be reasoned, evaluate the feature complexity, calculate the semantic entropy value of the input data, determine whether the current input data belongs to a simple sample, a medium-complex sample or a difficult sample, and allocate the initial computing resource quota. S5. The resource availability vector and the initial computational resource quota are calculated using the improved CMA-ES algorithm. A semantic entropy-guided population center initialization mechanism is introduced to map the initial search center of the population to the corresponding policy interval. Iterative sampling is performed in the policy solution space. The hardware overhead of the candidate policy is calculated using the hardware-aware cost model. The policy is then adaptively searched and output to the adaptive execution policy. S6. According to the adaptive execution strategy, call the corresponding computation branch in the initial lightweight model, perform high-precision floating-point operations on the critical layer, and perform low-precision fixed-point operations on the non-critical layer. Utilize forward inference propagation to output the feature map. S7. Perform feature decoupling and recovery on the feature map to compensate for the loss of detail caused by low-precision calculation or hierarchical truncation. After dequantization, output the final inference result data.

2. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S1 specifically includes: S11. Read the hardware specifications of the target low-resource device's memory, extract the memory bandwidth peak value, read the access timing data of different memory levels, obtain the access latency of different memory levels, read the thermal design power parameters in the processor specifications, determine the processor heat dissipation temperature threshold, and combine the memory bandwidth peak value, the access latency of different memory levels, and the processor heat dissipation temperature threshold into hardware physical characteristic parameters. S12. Preset a linear regression equation with computation time, energy consumption and temperature as independent variables. Substitute the peak memory bandwidth into the denominator coefficient of the computation time in the linear regression equation, substitute the access latency of different memory levels into the numerator coefficient of the computation time in the linear regression equation, and substitute the processor heat dissipation temperature threshold into the temperature constraint coefficient in the linear regression equation to establish a hardware-aware cost model.

3. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S2 specifically includes: S21. Create a neural architecture search algorithm that includes an encoder, decoder and long short-term memory network controller. Preset a search space consisting of attention mechanism layer, convolutional layer and pooling layer. Call the hardware-aware cost model to calculate the computational cost of the network architecture. S22. Initialize the hidden state vector of the Long Short-Term Memory Network Controller, set all elements of the hidden state vector to 0, input the start marker, obtain the position of the largest activation value in the output layer neuron, read the corresponding integer and assign it to the variable N, execute the loop, read the output vector of the first layer of the Long Short-Term Memory Network Controller, and take the index of the maximum value as the operation type of the current layer. S23. Execute the second loop, input the operation type of the first layer into the long short-term memory network controller, obtain the index of the second largest value as the operation type of the second layer, repeat the execution until the Nth loop ends, combine all the indexes obtained from the first to the Nth loop into a network architecture code in order, input the network architecture code into the encoder, and output the computation graph structure. S24. Read the parameter quantity of each layer in the computation graph structure, multiply the parameter quantity of each layer by the access latency of different memory levels to obtain the computation time of each layer, and add them together to obtain the total computation time. Read the number of floating-point operations of each layer in the computation graph structure, multiply the number of floating-point operations by the processor heat dissipation temperature threshold to obtain the energy consumption value of each layer, and add the energy consumption values ​​of all layers together to obtain the total energy consumption value. S25. If the total computation time exceeds the peak memory bandwidth or the total energy consumption exceeds the preset energy consumption limit, the current network architecture encoding is marked as invalid encoding. Otherwise, the neural network model corresponding to the current network architecture encoding is constructed, the training sample data is input to perform forward propagation, the error value between the output result and the label data is calculated, and the negative number is used to obtain the reward value. S26. Input the reward value into the Long Short-Term Memory Network Controller, update the weight matrix inside the Long Short-Term Memory Network Controller using the backpropagation algorithm, repeat the weight matrix update a preset number of times, extract the network architecture code with the largest reward value, and build the basic network model. S27. Read the weight values ​​of each connection between two neurons in the basic network model. If they are less than the preset weight threshold, delete the edge connecting the two neurons. Read the average value of all values ​​in each convolution kernel in the basic network model. If it is less than the preset average threshold, remove the convolution kernel from the network structure. Repeat the weight matrix update for the basic network model after removing the edge and convolution kernel a preset number of times to obtain the initial lightweight model.

4. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 3, characterized in that, The specific steps of constructing the neural network model corresponding to the current network architecture encoding include: Create an empty list structure to store network layer objects in order. Read the first value in the network architecture encoding sequence and determine if it is equal to 0. If it is equal to 0, preset the convolution kernel size, number of input channels, number of output channels, stride and padding. Instantiate a two-dimensional convolution operation class and append the instantiated object to the end of the list structure. If it is not equal to 0, then check if the value is equal to 1. If it is equal to 1, then instantiate a multi-head self-attention operation class with a preset embedding dimension size, number of attention heads and dropout rate, and append the instantiated object to the end of the list structure. Read the second value in the network architecture encoding sequence, repeat the judgment and append operation, and read sequentially until the object corresponding to the last value in the current network architecture encoding is instantiated and appended. Encapsulate the list structure containing all objects into the overall neural network model.

5. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S3 specifically includes: S31. Call the underlying performance monitoring interface of the target low-resource device operating system, read the current values ​​in the CPU utilization register and GPU utilization register respectively, record the current CPU value as CPU utilization, record the current GPU value as GPU utilization, read the number of free page addresses in memory management, multiply the number of free page addresses by the size of a single page, and calculate the current available memory balance. S32. Traverse the operating system process list, read the thread time slice of all processes except the current inference process, divide the sum of the time slices occupied by all background tasks by the total CPU clock cycles, calculate the proportion of computing resources occupied by the system background tasks, and record it as the utilization rate. S33. Read the core frequency value from the CPU's hardware specifications, multiply the core frequency value by 1 and subtract the CPU utilization value, then multiply by 1 and subtract the occupancy rate to calculate the maximum theoretical amount of computation that the CPU can currently handle. S34. Read the number of stream processors and core frequency values ​​from the GPU's hardware specifications, multiply the number of stream processors by the core frequency value, multiply by 1 and subtract the GPU utilization value to calculate the maximum theoretical amount of computation that the GPU can currently handle. S35. Arrange the current available memory, the maximum theoretical computational load that the CPU can currently handle, and the maximum theoretical computational load that the GPU can currently handle in sequence, and combine them in order to generate a real-time resource availability vector.

6. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S4 specifically includes: S41. Input the received input data to be inferred into the pre-trained Softmax classification layer, obtain the value of each output node, and sum them to obtain the denominator. Divide the value of each output node by the denominator to obtain the predicted probability value of each category. Call the logarithmic function to calculate the logarithmic value with the natural constant e as the base. Multiply the calculated logarithmic value by the predicted probability value of the corresponding category to obtain the entropy component of each category. Add the values ​​of the entropy components of all categories and record the sum as the semantic entropy value of the input data. S42. Compare the semantic entropy value with the preset entropy range threshold. If the semantic entropy value is less than the lower limit of the entropy range threshold, it is determined to be a simple sample, and the initial computing resource quota is set to 1. If it is within the entropy range threshold, it is determined to be a medium-complex sample, and the initial computing resource quota is set to 2. If it is greater than or equal to the upper limit of the entropy range threshold, it is determined to be a difficult sample, and the initial computing resource quota is set to 3.

7. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S5 specifically includes: S51. Read the real-time resource availability vector, create a three-dimensional coordinate system, set the horizontal axis label to the network depth truncation position, set the vertical axis label to the operator calculation precision bit width, and set the vertical axis label to the data flow path. Read the maximum number of layers at the network depth truncation position, the minimum and maximum bit values ​​of the operator calculation precision bit width, and the total number of data flow path numbers. S52. Set the maximum number of layers as the upper bound of the horizontal axis, set the minimum bit value and the maximum bit value as the lower bound and upper bound of the vertical axis, set the total number of numbers as the upper bound of the vertical axis, and set 0 as the lower bound of all coordinate axes to form a closed three-dimensional cube region as the strategy solution space. S53. Read the semantic entropy value, retrieve the numerical range where the semantic entropy value is located in the preset database, read the preset center point coordinate value corresponding to the numerical range, and assign it to the preset variable as the initial mean vector, and initialize the identity matrix of the preset number of dimensions as the initial covariance matrix. S54. Set the initial step size coefficient, call the random number generation function to generate a preset number of random values ​​that follow a standard normal distribution, generate a random value vector, and perform matrix multiplication with the covariance matrix. Multiply the result by the initial step size coefficient and add it to the initial mean vector to obtain the coordinates of an individual in the population. Repeat the operation of generating individual coordinates until the number of individuals reaches the preset number to obtain the initial population. S55. Traverse each individual in the initial population, read the horizontal axis coordinate value of the individual as the layer index that the network needs to retain, read the vertical axis coordinate value as the number of binary bits used when calculating the network layer, read the vertical axis coordinate value as the path number of data transmission between memory and processor, substitute the layer index, number of binary bits and path number into the hardware-aware cost model, and calculate the total computing time and total energy consumption value required for the current individual to run on the current device. S56. Subtract the total computation time of an individual from the peak memory bandwidth, and subtract the total energy consumption of an individual from the processor heat dissipation temperature threshold. If any result is positive, set the fitness value to 0. Otherwise, configure the layer index, number of bits, and path number corresponding to the current individual into the initial lightweight model, input validation sample data, count the number of correctly predicted results in the output results, divide by the total number of validation sample data, obtain the accuracy value, and use it as the fitness value of the current individual. S57. Sort the fitness of all individuals in the initial population from largest to smallest, select the first preset number of individuals and store them in the excellent set, add the horizontal axis coordinates of all individuals in the excellent set and divide by the total number of individuals in the excellent set to obtain the horizontal axis component of the new generation mean vector, use the same method to calculate the vertical axis component and the vertical axis component, and combine them to obtain the vertical axis component and the vertical axis component of the new generation mean vector. S58. Read the three-dimensional coordinate values ​​of each individual in the excellent set, subtract the coordinate components corresponding to the new generation mean vector to obtain the difference vector, perform matrix multiplication on all difference vectors and their corresponding transpose vectors, add the results of all individuals, divide the sum matrix by the total number of individuals in the excellent set to obtain the updated covariance matrix. S59. Read the initial calculation resource quota value and take its reciprocal to obtain the damping coefficient. Read the evolution path vector of the previous generation iteration. Subtract the mean vector of the new generation from the mean vector of the previous generation to obtain the mean difference vector and divide it by the square root matrix of the covariance matrix to obtain the evolution path vector of the current generation. Perform a weighted sum of the evolution path vector of the current generation and the evolution path vector of the previous generation with preset weights to obtain the updated evolution path. S510. Multiply the values ​​of the main diagonal elements of the covariance matrix by the damping coefficient to obtain the updated covariance matrix. Use the ratio of the magnitude of the updated evolution path vector to the preset expected length as the exponent to perform an exponential operation on the current step size coefficient to obtain a new step size coefficient. Use the updated mean vector, the updated covariance matrix and the new step size coefficient to generate the coordinates of the next generation of individuals through normal distribution sampling. S511. Using the next generation population as the current population, repeatedly execute the operations of reading coordinate values, calculating hardware overhead, comparison threshold, setting fitness values, calculating mean vector, calculating covariance matrix, updating step size coefficient, and generating coordinates of the next generation population. When the number of iterations equals the preset maximum number of iterations, read the horizontal, vertical, and vertical components of the mean vector in the last iteration, and use them as the final network depth truncation position, operator calculation precision bit width, and data flow path, respectively, and combine them in order as the output of the adaptive execution strategy for the current input data.

8. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S6 specifically includes: S61. Read the network depth truncation position value in the adaptive execution strategy, obtain all network layers in the initial lightweight model as a complete sequence, and traverse from the starting layer of the complete sequence according to the network depth truncation position value. Mark all network layers with layer index less than or equal to the network depth truncation position value as retained layers, and remove network layers with layer index greater than the network depth truncation position value from the inference process. S62. For each network layer in the reserved layer, read the operator calculation precision bit width value in the adaptive execution strategy and compare it with the preset precision threshold. If it is greater than or equal to the preset precision threshold, mark the current network layer as a critical layer; otherwise, mark the current network layer as a non-critical layer. S63. Traverse all network layers marked as key layers, perform floating-point operations on the target low-resource device, load the weight data of the key layers, perform matrix multiplication operations between the preset weight tensor and the input feature map using the preset floating-point number format, perform activation operations using the exponential and logarithmic functions, and output the high-precision feature map of the key layers. S64. Traverse all network layers marked as non-critical layers, perform fixed-point operations in the target low-resource device, read the data flow path in the adaptive execution strategy, convert the weight data and input data of non-critical layers into fixed-point number format according to the path instructions, perform dot product and accumulation operations using the preset integer format, and output low-precision feature maps of non-critical layers. S65. Following the connection order of the data flow in the initial lightweight model, the output feature map of the previous layer is used as the input feature map of the next layer. Matrix multiplication, dot product and accumulation operations are repeatedly performed until all retained layers are traversed. The output data of the last layer located at the network depth truncation position is obtained, which is the feature map after inference propagation.

9. The method for deploying and inferring large models in low-resource environments based on deep learning according to claim 1, characterized in that, S7 specifically includes: using a preset decoupling algorithm based on wavelet transform or dictionary learning to separate the low-bit quantized feature map in the frequency domain or sparse domain, reconstructing the image edge details and texture information lost due to low-precision truncation through high-frequency components, then performing an inverse quantization operation on the reconstructed feature map to accurately map the fixed-point values ​​back to the original floating-point domain, and outputting the final inference result data.

Images