A global sparsification method and device of a convolutional neural network model

Abstract

The application discloses a global sparsification method and device of a convolutional neural network model, and relates to the technical field of artificial intelligence. The method comprises the following steps: setting a constraint condition of a model; setting different random threshold values for each to-be-optimized convolution kernel channel through a Latin hypercube sampling algorithm; if the current model accuracy and the current model size do not satisfy the constraint condition, iteratively optimizing the threshold value of each convolution kernel channel through a differential evolution algorithm until a preset iteration number is reached, or the current model size and the current model accuracy satisfy the constraint condition; and repeatedly executing the model compression step until the current model accuracy and the current model size satisfy the constraint condition. The application has strong global search capability, good robustness, and can balance the model accuracy and the model size.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a global sparsification method and device of a convolutional neural network model. BACKGROUND

[0002] At present, with the rapid development of emerging applications such as AI artificial intelligence, the automobile industry, the Internet of Things, etc., the development of embedded AI MCUs is stimulated. In order to pursue low cost, cope with complex and fast AI tasks, especially image recognition tasks, end-side AI MCU applications have emerged. From the perspective of SOC chip design, the key storage and computing units of the system play a decisive role in chip cost, reaction speed, energy efficiency and other performances. For example, the current target detection convolutional neural network deployed in the end-side AI MCU is mainly the YOLO series model, and the YOLO series model is a popular target detection algorithm, which is famous for its speed and performance. The model is small but can simultaneously process target detection and give detection contours. The small YOLO model can reduce the computing and storage cost of the MCU chip. And compressing the YOLO model can greatly improve the energy efficiency of the MCU chip, reduce the chip size, and improve the speed of model inference.

[0003] In order to realize the compression or sparsification of the YOLO target detection model and similar convolutional neural network models, the prior art has adopted many schemes, such as model pruning, model sparse regularization, etc. However, the main shortcomings of the prior art are:

[0004] (1) The existing model sparsification scheme mainly focuses on how to set a threshold or reduce the model sparsity through a strategy, but the optimization of the threshold basically adopts a Bayesian optimization algorithm to find the optimal solution. Although the Bayesian optimization algorithm is fast, it is difficult to ensure global optimality due to the limitation of modeling ability, so a robust global optimization algorithm is needed to compress the model while also considering the model accuracy;

[0005] (2) The existing model sparsification scheme generally sets the same comparison threshold for the convolution kernel, which is difficult to perceive the importance of different convolution layers of the convolutional neural network model, resulting in unfair setting of the convolution kernel threshold, and the sparsified model is difficult to achieve global optimization. SUMMARY

[0006] In view of the above-mentioned defects or shortcomings in the prior art, the present application provides a global sparsification method and device of a convolutional neural network model, which can perfectly solve the technical problems existing in the background art.

[0007] In one aspect of the present application, a global sparsification method of a convolutional neural network model is provided, comprising: a constraint condition setting step of setting a constraint condition of model compression for a trained model, the constraint condition comprising a target model accuracy and a target model size; a model compression step of determining a number of convolution kernels to be optimized according to the target model size, setting different random thresholds for each convolution kernel channel to be optimized by a Latin hypercube sampling algorithm, performing a sparsification operation on the model based on the random thresholds, and calculating a current model accuracy and a current model size, if the current model accuracy and the current model size do not satisfy the constraint condition, iteratively evolving a new threshold for each convolution kernel channel by a differential evolution algorithm, and performing the sparsification operation on the model based on the new threshold evolved in each generation until a preset iteration number is reached, or the current model size and the current model accuracy satisfy the constraint condition; wherein the sparsification operation comprises deleting a convolution kernel when the number of weights and bias terms of the convolution kernel that are zero is greater than the threshold; and a loop execution step of repeatedly performing the model compression step until the current model accuracy and the current model size satisfy the constraint condition if the current model accuracy and the current model size do not satisfy the constraint condition.

[0008] In another aspect of the present application, a global sparsification device of a convolutional neural network model is also provided, comprising: a constraint condition setting module for setting a constraint condition of model compression for a trained model, the constraint condition comprising a target model accuracy and a target model size; a model compression module for determining a number of convolution kernels to be optimized according to the target model size, setting different random thresholds for each convolution kernel channel to be optimized by a Latin hypercube sampling algorithm, performing a sparsification operation on the model based on the random thresholds, and calculating a current model accuracy and a current model size, if the current model accuracy and the current model size do not satisfy the constraint condition, iteratively evolving a new threshold for each convolution kernel channel by a differential evolution algorithm, and performing the sparsification operation on the model based on the new threshold evolved in each generation until a preset iteration number is reached, or the current model size and the current model accuracy satisfy the constraint condition; wherein the sparsification operation comprises deleting a convolution kernel when the number of weights and bias terms of the convolution kernel that are zero is greater than the threshold; and a loop execution module for repeatedly performing the steps in the model compression module until the current model accuracy and the current model size satisfy the constraint condition if the current model accuracy and the current model size do not satisfy the constraint condition.

[0009] The global sparsity reduction method and apparatus for convolutional neural network models provided by this invention first initializes the thresholds of multiple convolutional kernels using the Latin hypercube sampling algorithm. Different random thresholds are used to perceive the importance of different layers in the convolutional neural network model, forming relatively fair thresholds. Then, the differential evolution algorithm is used to optimize the thresholds in each channel of the convolutional kernel to obtain the optimal model size while ensuring model accuracy to guarantee inference capability. The global sparsity reduction method and apparatus for convolutional neural network models of this invention have strong global search capabilities and good robustness, and can balance model accuracy and model size. Attached Figure Description

[0010] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0011] Figure 1 This is a flowchart illustrating a global sparsification method for a convolutional neural network model provided in one embodiment of this application. Figure 1 ;

[0012] Figure 2 This is a flowchart illustrating a global sparsification method for a convolutional neural network model provided in one embodiment of this application. Figure 2 ;

[0013] Figure 3 This is a schematic diagram of the structure of a global sparsification device for a convolutional neural network model provided in one embodiment of this application;

[0014] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0017] It should be understood that although the terms first, second, third, etc. can be employed in describing the obtaining modules in the embodiments of the present application, the obtaining modules should not be limited to these terms. These terms are only used to distinguish the obtaining modules from each other.

[0018] Depending on context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "when it is determined" or "in response to determining" or "when [a stated condition or event] is detected" or "in response to detecting [a stated condition or event]."

[0019] It should be noted that the terms "upper", "lower", "left", "right", etc. described in the embodiments of the present application are described in the angle shown in the drawings, and should not be understood as a limitation on the embodiments of the present application. In addition, in the context, it should also be understood that when referring to an element formed "on" or "under" another element, it can be directly formed "on" or "under" another element, or indirectly formed "on" or "under" another element through an intermediate element.

[0020] In view of the requirements of high energy efficiency and fast response of the end-side AI MCU, the size of the convolutional neural network model such as the target detection network needs to be compressed, and the most common way of model compression is to perform model sparsification processing. An embodiment of the present application takes the sparsification of the YOLO v5n target detection network model as an example, and proposes a global sparsification method and device of a convolutional neural network model.

[0021] Referring to Figure 1 , 2 The global sparsification method of the convolutional neural network model proposed in the embodiment includes the following steps:

[0022] Step S101, constraint condition setting step: setting the constraint condition of model compression for the trained model, and the constraint condition includes target model accuracy and target model size.

[0023] First, according to the application scenario of the YOLO model, an image data set is prepared. For example, 200 video data in road driving are collected. The image data in the video are sorted, and a total of 8000 data are sorted. The learning target in the data is labeled to form a data set The data set is divided according to the ratio of training, verification and testing = 8:1:1.

[0024] Then, the YOLO official v5n model is downloaded, which has been trained, but can be retrained on new tasks, for example: training with GPU, the whole environment is based on Pytorch environment and python environment, setting the model evaluation function as BCE function and CIoU function, setting the initial model learning rate as 0.0001, epoch as 10000, using cosine learning rate reduction strategy. In order to prevent overfitting, the preferred early stopping strategy is introduced to avoid overfitting. During training, the model is verified on the validation dataset and tested on the test dataset, and the model accuracy is obtained after weighting the loss function . The model is saved as YOLOv5n.pt, and the model size is .

[0025] Then, the target model accuracy and target model size are determined, and the optimization task is to minimize the model size under the premise of meeting the target model accuracy. For example, set the target F(x) as the model size, and the constraint , that is, the maximum decline of model accuracy is 5%.

[0026] Step S102, model compression step: according to the target model size, the number of convolution kernels to be optimized is determined, different random thresholds are set for each convolution kernel channel to be optimized through the Latin hypercube sampling algorithm, the model is sparsified based on the random threshold, and the current model accuracy and the current model size are calculated. If the current model accuracy and the current model size do not meet the constraint condition, the difference evolution algorithm is used to iteratively evolve new threshold values for each convolution kernel channel, and the sparsification operation of the model is performed based on the new threshold values evolved in each generation until a preset iteration number is reached, or the current model size and the current model accuracy meet the constraint condition; wherein the sparsification operation includes deleting the convolution kernel when the number of weights and bias terms of the convolution kernel is greater than the threshold value.

[0027] Specifically, different random thresholds are set for each convolution kernel channel to be optimized through the Latin hypercube sampling algorithm, including: according to the target model size and experience, the number of convolution kernels to be optimized is selected, or all convolution kernels can be directly set as the optimization target. Then, different random thresholds are set for each convolution kernel channel to be optimized through the Latin hypercube sampling algorithm. The threshold value here represents the comparison threshold value of the total number of 0 in the convolution kernel weight and bias term. Specifically, the threshold value is set through the following steps:

[0028] Step one: given convolution kernel objects to be optimized, the threshold value range of the i-th dimension of each convolution kernel to be optimized is set as ];

[0029] Step two: divide the i-th dimension of each convolution kernel to be optimized into N intervals, and the threshold length of each interval is recorded as: ;

[0030] Step three: set a random sampling probability for each interval of the i-th dimension of each convolution kernel to be optimized , wherein represents the random sampling probability of the N-th interval of the i-th dimension;

[0031] Step four: calculate the sample threshold of the j-th interval on the i-th dimension of each convolution kernel to be optimized according to the following formula: , wherein represents the random sampling probability of the j-th interval of the i-th dimension, [0, N];

[0032] Step five: take the N sample thresholds on each dimension of each convolution kernel to be optimized as the threshold respectively.

[0033] The Latin hypercube sampling algorithm is used to perform hyper-mean initialization on the convolution kernel threshold. Different random thresholds are used for different convolution layers to perceive the importance of different layers of the model and form a more fair threshold. After setting different thresholds for the convolution kernels to be optimized, the model is trained and it is judged whether the total number of 0 or values very close to 0 (0 is an invalid calculation) in the convolution kernel weight and the bias term is greater than the set threshold. If yes, it means that the convolution kernel does not participate in subsequent operation, and the convolution kernel is deleted (i.e. sparse processing). After the sparse operation, the current model is saved and the current model accuracy and current model size are calculated. If the current model accuracy and the current model size meet the constraint condition, the optimized model is output; if the current model accuracy and the current model size do not meet the constraint condition, the threshold of each convolution kernel channel is iteratively optimized by the differential evolution algorithm.

[0034] Specifically, the threshold of each convolution kernel channel is iteratively optimized by the differential evolution algorithm, including the following steps:

[0035] Step one, set the population algebra to be iterated , the number of populations , , set as the model size, the problem is oriented to minimize the problem, set the constraint model accuracy, consider convolution kernel objects to be optimized, all convolution kernels form a design sample , set the threshold range of each convolution kernel to be [0, 1000];

[0036] ​Step two, use the Latin hypercube sampling algorithm to randomly initialize the population, starting from [ The average number of Chinese Super League teams generated The sample threshold is used to evaluate the sparsified model size and model accuracy when each sample threshold is calculated as the threshold.

[0037] Step 3: Generate the next generation of population through mutation, crossover, and selection operations.

[0038] Specifically, mutation operations are performed on the parent population based on the following formula:

[0039]

[0040] in, It is a scaling factor between 0 and 2; and Thresholds for two random samples; This represents the sample threshold that minimizes the model while satisfying the model accuracy constraints, or the sample threshold that makes the model closest to the constraints without satisfying the model accuracy constraints.

[0041] Specifically, cross-operations are performed through subordinates:

[0042] Dimension of the threshold for the i-th sample Using binomial cross, utilizing each dimension Generated random numbers are used to perform mutations:

[0043]

[0044] in, Indicates the first In a population of generations, CR represents the crossover probability. Represents a uniformly random number between 0 and 1. Indicates the random dimension. Indicates the first digit before mutation The generation The first Dimensional sample threshold, Indicates the mutated th The generation The first Dimensional sample threshold, Indicates the number after the cross The generation The first Dimensional sample threshold;

[0045] Specifically, the selection operation is performed in the following manner:

[0046] performing a sparsification operation and model evaluation, and selecting the next generation of the population to be evaluated according to the following formula :

[0047]

[0048] wherein, represents a target function of the model, represents the i-th sample threshold value in the G-th generation after crossover, represents the i-th sample threshold value in the G-th generation before mutation. represents the i-th sample threshold value in the G-th generation before mutation.

[0049] Step four, repeating the population evolution process until a preset iteration number of the population is reached, or the current model size and the current model accuracy meet the constraint condition.

[0050] until a preset iteration number is reached, or the current model size and the current model accuracy meet the constraint condition. Exemplarily, the parameters of the differential evolution algorithm are set as 60, the scaling factor F is set as 0.7, the crossover probability CR is set as 0.5, and the population generation G is set as 100. When the model accuracy meets the requirement and the model size reaches the size of B (B is the model size), the optimization process is stopped. The pruned model is output and saved as YOLOv5n_new.pt.

[0051] Step S103, repeatedly performing the following steps: if the current model accuracy and the current model size do not meet the constraint condition, then repeatedly performing the model compression step until the current model accuracy and the current model size meet the constraint condition.

[0052] Specifically, after the preset iteration number of the population is reached, the model still does not meet the constraint condition. At this time, step S102 needs to be re-executed, that is, the initial random threshold value is re-set by using the Latin hypercube sampling algorithm, and then the differential evolution algorithm is used to iteratively optimize each channel threshold value. In the evolution process of each generation of threshold values, sparsification processing is also performed based on the new threshold value until the current model accuracy and the current model size meet the constraint condition.

[0053] Further, the global sparsification method of the convolutional neural network model of the embodiment further comprises a model fine-tuning step for fine-tuning the architecture parameters of the sparsification-processed model to improve the model accuracy.

[0054] ​​​​​Specifically, the model is fine-tuned based on the sparse processing model, because the accuracy of the model after sparsification will inevitably decrease, and fine-tuning the model architecture parameters can further improve the model accuracy. The learning rate should be controlled during fine-tuning to prevent the model from changing significantly and further improve the model accuracy. Preferably, an early stopping strategy is adopted during model fine-tuning to prevent overfitting. Exemplarily, the final model accuracy is , the model size is , and the saved model size is YOLOv5n_small.pt.

[0055] The model is deployed in an embedded AI MCU for target detection tasks. The compressed model after sparsification is small but has high accuracy, which can reduce the computational requirements of the MCU, thereby reducing the design pressure of the MCU chip, reducing the chip size, and improving the chip energy efficiency ratio.

[0056] The global sparsification method and device of the convolutional neural network model provided by the application first initializes multiple convolution kernel thresholds using a Latin hypercube sampling algorithm, uses different random thresholds to perceive the importance of different layers of the convolutional neural network model, and forms relatively fair thresholds. Then, the difference evolution algorithm is used to optimize the threshold in each channel of the convolution kernel to obtain the best model size while ensuring the model accuracy to ensure the inference ability. The global sparsification method and device of the convolutional neural network model provided by the application have strong global search ability and good robustness, and can balance the model accuracy and model size.

[0057] Referring to Figure 3 , another embodiment of the application further provides a global sparsification device 200 of a convolutional neural network model, comprising a constraint condition setting module 201, a model compression module 202, and a loop execution module 203. The global sparsification device 200 of the convolutional neural network model can execute the global sparsification method of the convolutional neural network model in the method embodiment.

[0058] Specifically, the global sparsification device 200 of the convolutional neural network model comprises:

[0059] The constraint condition setting module 201 is configured to set the constraint conditions for model compression for the trained model, and the constraint conditions include the target model accuracy and the target model size.

[0060] The model compression module 202 is used to determine the number of convolutional kernels to be optimized based on the target model size, set different random thresholds for each channel of the convolutional kernel to be optimized using the Latin hypercube sampling algorithm, perform sparsification on the model based on the random thresholds, and calculate the current model accuracy and current model size. If the current model accuracy and current model size do not meet the constraints, a new threshold for each convolutional kernel channel is iteratively evolved using a differential evolution algorithm. The sparsification operation of the model is performed based on the new thresholds evolved in each generation until a preset number of iterations is reached, or the current model size and current model accuracy meet the constraints. The sparsification operation includes deleting the convolutional kernel when the number of zero weights and bias terms of the convolutional kernel is greater than the threshold.

[0061] The loop execution module 203 is used to repeatedly execute the steps in the model compression module until the current model accuracy and current model size meet the constraints if the current model accuracy and current model size do not meet the constraints.

[0062] Furthermore, it also includes a model fine-tuning module, used to fine-tune the architectural parameters of the sparsified model to improve model accuracy. This model fine-tuning module employs an early stopping strategy to prevent overfitting during fine-tuning.

[0063] It should be noted that the global sparsification device 200 of the convolutional neural network model provided in this embodiment corresponds to the technical solution that can be used to execute various method embodiments. Its implementation principle and technical effect are similar to the method, and will not be repeated here.

[0064] See Figure 4 Another embodiment of the present invention provides a schematic diagram of the structure of an electronic device 300, which is used to implement the global sparsification method of the convolutional neural network model in the method embodiment. The electronic device 300 in the embodiments of the present invention may include, but is not limited to, smartphones, tablet computers, PCs, laptops, and servers. Figure 4 The electronic device 300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0065] like Figure 4As shown, the electronic device 300 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301 that can perform various appropriate actions and processes to implement the methods of the embodiments as described in the present application according to programs stored in a read-only memory (ROM) 302 or loaded into a random access memory (RAM) 303 from a storage device 308. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 305. An input / output (I / O) interface 304 is also connected to the bus 305.

[0066] Generally, the following devices can be connected to the I / O interface 304: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 can allow the electronic device 300 to communicate wirelessly or wired with other devices to exchange data. Although Figure 4 The electronic device 300 is shown with various devices, but it should be understood that not all of the shown devices are required to be implemented or present. More or fewer devices can alternatively be implemented or present.

[0067] The above description is merely that of the preferred embodiments of the application. It is to be understood that the scope of the disclosure is not limited to the particular details described above but encompasses various alternatives and modifications. For example, the features described above and the features disclosed in the present application (but not limited to) having similar functions can be mutually substituted to form other technical solutions.

Claims

1. A global sparsification method for convolutional neural network models used in edge AI MCUs, for electronic devices, characterized in that... The method comprises the following steps: A model training step: collecting video data in road driving, organizing image data in the video, labeling learning targets in the data to form a data set, dividing the data set according to a ratio of training, verification and testing = 8:1:1, selecting a GPU to train a YOLO v5n target detection network model, and obtaining a trained model; A constraint condition setting step: setting constraint conditions for model compression for the trained model, wherein the constraint conditions comprise target model accuracy and target model size; A model compression step: determining the number of convolution kernels to be optimized according to the target model size, setting different random threshold values for each convolution kernel channel to be optimized by using a Latin hypercube sampling algorithm, performing a sparse operation on the model based on the random threshold values, calculating current model accuracy and current model size, and if the current model accuracy and the current model size do not meet the constraint conditions, iteratively evolving new threshold values for each convolution kernel channel by using a differential evolution algorithm, performing a sparse operation on the model based on the new threshold values evolved in each generation, until a preset iteration number is reached, or the current model size and the current model accuracy meet the constraint conditions; wherein the sparse operation comprises deleting a convolution kernel when the number of weights and bias terms of the convolution kernel is greater than the threshold value; A loop execution step: if the current model accuracy and the current model size do not meet the constraint conditions, repeating the model compression step until the current model accuracy and the current model size meet the constraint conditions; A deployment execution step: deploying the model into an embedded AI MCU.

2. The global sparsification method for a convolutional neural network model of an end-side AI MCU according to claim 1, characterized in that, The step of setting different random threshold values for each convolution kernel channel to be optimized by using a Latin hypercube sampling algorithm comprises: set the threshold range of the first dimension of each convolution kernel to be optimized as [0, 255]; i ];​ Divide the first dimension of each convolution kernel to be optimized into N intervals, and the threshold length of each interval is recorded as: i ; and ; set a random sampling probability for each interval of each dimension of each convolution kernel to be optimized i :​ ； wherein, represents the random sampling probability of the Nth interval of the Mth dimension. i represents the random sampling probability of the Nth interval of the Mth dimension. The following formula is used to calculate the first convolutional kernel to be optimized. i The first dimension j Sample threshold for each interval : ； wherein, denotes the dimension i of the first j interval of the random sampling probability, [0,N]; Respectively taking N sample threshold values in each dimension of each convolution kernel to be optimized as the random threshold values.

3. The method of claim 2, wherein, The step of iteratively evolving new threshold values for each convolution kernel channel by using a differential evolution algorithm, performing a sparse operation on the model based on the new threshold values evolved in each generation, until a preset iteration number is reached, or the current model size and the current model accuracy meet the constraint conditions, comprises: Generating a plurality of sample threshold values as a parent population from a threshold value range of each convolution kernel by using a Latin hypercube sampling algorithm; Performing a mutation operation on the parent population based on the following formula: wherein, is a scaling factor between 0 and 2; and are two random sample thresholds; denotes the sample threshold that minimizes the model under the satisfaction of the model accuracy constraint, or the sample threshold that is closest to the constraint condition under the non-satisfaction of the model accuracy constraint; Regarding the first i Dimension of threshold for each sample Using binomial cross, utilizing each dimension Generated random numbers are used to perform mutations: in, Indicates the first In a population of generations, CR represents the crossover probability. Represents a uniformly random number between 0 and 1. Indicates the random dimension. Indicates the first digit before mutation The generation The first Dimensional sample threshold, Indicates the mutated th The generation The first Dimensional sample threshold, Indicates the number after the cross The generation The first Dimensional sample threshold; performing a sparsification operation and model evaluation, selecting a next generation of population to be evaluated according to the following equation : wherein, a target function representing the model, represents the threshold of the i-th sample after the crossover, represents the threshold of the i-th sample after the mutation, represents the threshold of the i-th sample after the crossover, represents the threshold of the i-th sample after the mutation, represents the threshold of the i-th sample after the crossover, represents the threshold of the i-th sample after the mutation, Repeating the population evolution process until a preset iteration number of the population is reached, or the current model size and the current model accuracy meet the constraint conditions.

4. The global sparsification method for a convolutional neural network model of an end-side AI MCU according to claim 3, characterized in that, The method further comprises a model fine-tuning step for fine-tuning the architecture parameters of the sparse processed model to improve the model accuracy.

5. The global sparsification method for a convolutional neural network model of an end-side AI MCU according to claim 4, characterized in that, An early stopping strategy is adopted in the model fine-tuning process to prevent overfitting.

6. A device for global sparsification of a convolutional neural network model for end-side AI MCUs, for use in electronic devices, characterized in that The method comprises the following steps: A model training module for collecting video data in road driving, organizing image data in the video, labeling learning targets in the data to form a data set, dividing the data set according to a ratio of training, verification and testing = 8:1:1, selecting a GPU to train a YOLO v5n target detection network model, and obtaining a trained model; The constraint setting module is configured to set a constraint condition for model compression for the trained model, the constraint condition including a target model accuracy and a target model size. The model compression module is configured to determine the number of convolution kernels to be optimized according to the target model size, set different random threshold values for each convolution kernel channel to be optimized by using a Latin hypercube sampling algorithm, perform a sparse operation on the model based on the random threshold values, and calculate a current model accuracy and a current model size. If the current model accuracy and the current model size do not meet the constraint condition, the model compression module is further configured to evolve a new threshold value for each convolution kernel channel by using a differential evolution algorithm, perform the sparse operation on the model based on the new threshold value evolved in each generation, until a preset iteration number is reached, or the current model size and the current model accuracy meet the constraint condition. The sparse operation includes deleting a convolution kernel when the number of weight and bias items of the convolution kernel is greater than the threshold value. The loop execution module is configured to repeatedly perform the model compression step until the current model accuracy and the current model size meet the constraint condition. The deployment execution module is configured to deploy the model to an embedded AI MCU.

7. The device for global sparsification of a convolutional neural network model for end-side Al MCUs according to claim 6, characterized in that, The model compression module is further configured to: set the threshold range of the first dimension of each convolution kernel to be optimized as [0, 255]; i ]​ Divide the first dimension of each convolution kernel to be optimized into N intervals, and the threshold length of each interval is recorded as: i ; and ; set a random sampling probability for each interval of each dimension of each convolution kernel to be optimized i :​ ； wherein, represents the Nth interval of the Mth dimension; i represents the random sampling probability of the Nth interval of the Mth dimension; The sample threshold of the i-th interval on the j-th dimension of each convolution kernel to be optimized is calculated according to the following formula: i : sample threshold of the i-th interval on the j-th dimension of the convolution kernel to be optimized j : sample threshold of the i-th interval on the j-th dimension of the convolution kernel to be optimized : sample threshold of the i-th interval on the j ； wherein, denotes the dimension i of the first j interval of the random sampling probability, [0,N]; use N sample threshold values in each dimension of each convolution kernel to be optimized as the random threshold values.

8. The device for global sparsification of a convolutional neural network model for end-side Al MCUs according to claim 7, characterized in that, The model compression module is further configured to: generate a plurality of sample threshold values as a parent population from a threshold value range of each convolution kernel by using a Latin hypercube sampling algorithm; perform a mutation operation on the parent population based on the following formula: wherein, is a scaling factor between 0 and 2; and are two random sample thresholds; denotes the sample threshold that minimizes the model under the satisfaction of the model accuracy constraint, or the sample threshold that is closest to the constraint condition under the non-satisfaction of the model accuracy constraint; Regarding the first i Dimension of threshold for each sample Using binomial cross, utilizing each dimension Generated random numbers are used to perform mutations: in, Indicates the first In a population of generations, CR represents the crossover probability. Represents a uniformly random number between 0 and 1. Indicates the random dimension. Indicates the first digit before mutation The generation The first Dimensional sample threshold, Indicates the mutated th The generation The first Dimensional sample threshold, Indicates the number after the cross The generation The first Dimensional sample threshold; performing a sparsification operation and model evaluation, selecting a next generation of population to be evaluated according to the following equation : wherein, a target function representing the model, a threshold of the i-th sample in the j-th generation after crossover, a threshold of the i-th sample in the j-th generation before mutation, a threshold of the i-th sample in the j-th generation after mutation, a threshold of the i-th sample in the j-th generation before crossover, a threshold of the i-th sample in the j-th generation before mutation, a threshold of the i-th sample in the j-th generation after mutation, repeat the population evolution process until a preset iteration number of the population is reached, or the current model size and the current model accuracy meet the constraint condition.

9. The device for global sparsification of a convolutional neural network model for end-side Al MCUs according to claim 8, characterized in that, The model fine-tuning module is further configured to fine-tune the architecture parameters of the model after the sparse operation to improve the model accuracy.

10. The device for global sparsification of a convolutional neural network model for end-side Al MCUs according to claim 9, characterized in that, The model fine-tuning module adopts an early stopping strategy to prevent overfitting during fine-tuning.

Images