Training method, device and server for image detection model deployed on terminals

Abstract

The present application discloses a training method, device, and server for an image detection model deployed on a terminal, belonging to the field of computer technology. The method comprises: obtaining an image detection model to be deployed on the terminal, processing the image detection model to obtain a remodeled image detection model comprising a single-branch structure; training the remodeled image detection model using a gradient mask method, quantizing the trained remodeled image detection model, and, if the accuracy of the quantized remodeled image detection model is less than a preset accuracy, performing quantization-perceptual training on the trained remodeled image detection model using a step-size learnable method and a feature distillation method to obtain a trained remodeled image detection model; and deploying the trained remodeled image detection model on the terminal. The accuracy of the remodeled image detection model after quantization-perceptual training is ensured, and the training process of the image detection model with a single-branch structure is fast, thereby improving training efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a training method, device, and server for an image detection model for terminal deployment. Background Art

[0002] With the rapid development of computer technology, a variety of model structures have been developed to enable models with different functions and applications in different environments. Because the data volume of normal models is too large, directly deploying the models on terminals will cause the terminals to be unable to operate the models properly. Therefore, a method for quantizing the models is proposed to compress the data volume.

[0003] Specifically, an image detection model is proposed. This model uses a multi-branch structure during training. During deployment, the multi-branch structure is merged to quantize the image detection model. The quantized model is then deployed on the terminal to ensure normal operation of the model. However, the quantization of this image detection model uses a small amount of data, which reduces the accuracy of the image detection model and fails to meet the accuracy requirements for detection by the terminal using this image detection model. Summary of the Invention

[0004] The present invention provides a training method, device, and server for an image detection model deployed on a terminal. These methods ensure the accuracy of the modified image detection model after quantitative perceptual training. Furthermore, the training process of the image detection model with a single-branch structure is fast, thereby improving training efficiency. The technical solution is as follows:

[0005] In one aspect, a method for training an image detection model for terminal deployment is provided, the method comprising:

[0006] Obtain an image detection model to be deployed on the terminal, wherein the image detection model is used to perform target detection on the image, and the image detection model includes multiple branch structures;

[0007] Merging the multiple branch structures of the image detection model to obtain a modified image detection model including a single branch structure;

[0008] Based on the sample image set, the transformed image detection model is trained using a gradient mask method, and the trained transformed image detection model is quantized to obtain a quantized transformed image detection model;

[0009] When the accuracy of the quantized reshaped image detection model is less than a preset accuracy, based on the sample image set, the trained reshaped image detection model is subjected to quantization-aware training using a step-size learnable method and a feature distillation method to obtain a trained reshaped image detection model, wherein the step-size learnable method refers to supporting updates of the quantization step size when the reshaped image detection model is quantized, and the feature distillation method refers to training the quantized reshaped image detection model using the reshaped image detection model before quantization;

[0010] The trained transformed image detection model is deployed on the terminal.

[0011] In one aspect, a training apparatus for an image detection model for terminal deployment is provided, the apparatus comprising:

[0012] An acquisition module is used to acquire an image detection model to be deployed on the terminal, wherein the image detection model is used to perform target detection on the image, and the image detection model includes multiple branch structures;

[0013] a merging module, configured to merge the multiple branch structures of the image detection model to obtain a modified image detection model including a single branch structure;

[0014] A training module, configured to train the reshaped image detection model based on a sample image set using a gradient mask method, and quantize the trained reshaped image detection model to obtain a quantized reshaped image detection model;

[0015] The training module is further configured to, when the accuracy of the quantized reshaped image detection model is less than a preset accuracy, perform quantization-aware training on the trained reshaped image detection model based on the sample image set using a step-size learnable method and a feature distillation method to obtain a trained reshaped image detection model, wherein the step-size learnable method refers to supporting updates of the quantization step size when quantizing the reshaped image detection model, and the feature distillation method refers to training the quantized reshaped image detection model using the reshaped image detection model before quantization;

[0016] A deployment module is used to deploy the trained transformed image detection model to the terminal.

[0017] In a possible implementation, the merging module is configured to:

[0018] For each branch structure of the plurality of branch structures, expanding a weight matrix corresponding to the branch structure to obtain an expanded weight matrix of a preset size;

[0019] The extended weight matrix corresponding to each branch structure is weighted to obtain a reconstructed image detection model including the single branch structure.

[0020] In a possible implementation, the training module is used to:

[0021] Obtaining a scaling factor of the reconstructed image detection model, wherein the scaling factor is used to quantize the reconstructed image detection model;

[0022] Based on the weights of the reconstructed image detection model and the scaling factor, updating the weights and the scaling factor of the reconstructed image detection model;

[0023] Based on the trained transformed image detection model and the updated weights and scaling factors, the sample image set is detected to obtain a prediction result;

[0024] Based on the difference between the sample result corresponding to the sample image set and the prediction result, the trained reconstructed image detection model is trained to obtain the trained reconstructed image detection model.

[0025] In a possible implementation, the training module is used to:

[0026] Obtaining a first ratio of an activation value of the transformed image detection model to the scaling factor and a second ratio of the weight to the scaling factor;

[0027] Rounding the first ratio and the second ratio to integers respectively to obtain a first value corresponding to the first ratio and a second value corresponding to the second ratio;

[0028] Determine a product of the first value and the scaling factor and a product of the second value and the scaling factor as a pseudo-quantized activation value and a weight;

[0029] Calculate the loss value based on the pseudo-quantized activation value, the pseudo-quantized weight and the scaling factor;

[0030] Backpropagate the loss value to obtain updated weights and scaling factors.

[0031] In a possible implementation, the training module is used to:

[0032] The transformed image detection model before quantization is used as the teacher model, and the transformed image detection model after quantization is used as the student model;

[0033] Based on the teacher model and the student model, target detection is performed on the sample image set to obtain a first feature matrix and a first detection result output by the teacher model, and a second feature matrix and a second detection result output by the student model;

[0034] Based on the difference between the first feature matrix and the second feature matrix, and the difference between the first detection result and the second detection result, the student model is trained to obtain a trained transformed image detection model.

[0035] In a possible implementation, the training module is used to:

[0036] When the accuracy of the quantized reshaped image detection model is less than a preset accuracy, performing quantization perception training on the trained reshaped image detection model based on the sample image set using the step-size learnable method;

[0037] When the trained transformed image detection model is quantitatively trained using the step-size learnable method and the accuracy of the transformed image detection model obtained is less than the preset accuracy, the transformed image detection model is quantitatively trained using the feature distillation method.

[0038] In a possible implementation, the acquisition module is used to acquire multiple model layers of the trained transformed image detection model;

[0039] The device also includes: the fusion module, which is used to fuse the model layers of the same type when there are model layers of the same type in the multiple model layers, to obtain a fused transformed image detection model, and the accuracy of the fused transformed image detection model is the same as the accuracy of the transformed image detection model after training.

[0040] In a possible implementation, the model layers of the same type refer to model layers having a linear relationship, or the model layers of the same type refer to model layers having the same structure.

[0041] On the one hand, a server is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by the training method of the image detection model for terminal deployment in any possible implementation as described above.

[0042] On the one hand, a computer-readable storage medium is provided, which stores at least one program code, which is loaded and executed by a processor to implement the operations performed by the training method of the image detection model for terminal deployment in any possible implementation as described above.

[0043] On the one hand, a computer program or computer program product is provided, which includes: computer program code, which, when executed by a server, enables the server to implement the operations performed by the training method for an image detection model for terminal deployment in any of the possible implementations described above.

[0044] The training method of the image detection model for terminal deployment provided in the embodiment of the present application transforms the image detection model with a multi-branch structure into a single-branch structure, and then simultaneously adopts a step-size learnable method and a feature distillation method to perform quantitative perception training on the transformed image detection model to ensure the accuracy of the transformed image detection model after the quantitative perception training. After the transformed image detection model after the quantitative perception training is deployed on the terminal, the structure of the model is quantized so that the terminal can operate the transformed image detection model normally, and the completed training ensures the accuracy requirements of the terminal using the transformed image detection model for detection, and the training process of the image detection model with a single-branch structure is fast, thereby improving the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0047] Figure 2 This is a flowchart of a training method for an image detection model deployed on a terminal provided in an embodiment of the present application;

[0048] Figure 3 This is a flowchart of a training method for an image detection model deployed on a terminal provided in an embodiment of the present application;

[0049] Figure 4 This is a structural diagram of a modified image detection model provided in an embodiment of the present application;

[0050] Figure 5 This is a flowchart of a method for updating a scaling factor provided by an embodiment of the present application;

[0051] Figure 6 This is a flow chart of a characteristic distillation method provided in an embodiment of the present application;

[0052] Figure 7 This is a schematic diagram of another model layer fusion provided in an embodiment of the present application;

[0053] Figure 8 This is a schematic diagram of a model layer fusion provided in an embodiment of the present application;

[0054] Figure 9 This is a flowchart of a training process of an image detection model for terminal deployment provided by an embodiment of the present application;

[0055] Figure 10 1 is a schematic structural diagram of a training device for an image detection model deployed on a terminal, provided in an embodiment of the present application;

[0056] Figure 11 is a schematic diagram of the structure of the terminal provided in an embodiment of the present application;

[0057] Figure 12 It is a structural diagram of the server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0059] It is understood that the terms "first," "second," and the like used herein may be used to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are used solely to distinguish one concept from another. For example, a first characteristic matrix may be referred to as a second characteristic matrix, and a second characteristic matrix may be referred to as a first characteristic matrix, without departing from the scope of this application.

[0060] The terms "at least one", "multiple", "each", and "any" used in this application, at least one includes one, two or more than two, multiple includes two or more than two, and each refers to each of the corresponding multiple, and any refers to any one of the multiple. For example, multiple branch structures include 3 branch structures, and each refers to each branch structure of the 3 branch structures, and any refers to any one of the 3 branch structures, which can be the first, the second, or the third.

[0061] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the positioning information involved in this application is obtained with full authorization. Moreover, the above information and data are processed and used in big data application scenarios, and cannot be identified to any natural person or have a specific association with him or her.

[0062] In some embodiments, the training method of the image detection model provided in the embodiments of the present application is executed by a terminal.

[0063] In other embodiments, the training method of the image detection model provided in the embodiments of the present application is performed by a terminal and a server. The server can be a single server, a server cluster consisting of several servers, or a cloud computing service center.

[0064] It should be noted that the embodiments of the present application do not limit the execution entity of the training method of the image detection model.

[0065] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application, such as Figure 1 As shown, the implementation environment includes a terminal 101 and a server 102, and the terminal 101 and the server 102 are connected via a wireless or wired network.

[0066] Among them, the server 102 is a server that provides services related to model training for the terminal 101. In some embodiments, the terminal 101 captures an image and sends the captured image to the server 102. The server 102 serves as a sample image set. Then, the server 102 obtains an image detection model to be deployed on the terminal. The image detection model is used to detect targets on the image, and the image detection model includes multiple branch structures. The multiple branch structures of the image detection model are merged to obtain a reconstructed image detection model including a single branch structure. Based on the sample image set, the reconstructed image detection model is trained using a gradient mask method, and the trained reconstructed image detection model is quantized to obtain a quantized reconstructed image detection model. When the accuracy of the quantized reconstructed image detection model is less than Under the preset accuracy, based on the sample image set, the trained transformed image detection model is trained with a step-length learnable method and a feature distillation method to obtain a trained transformed image detection model. The step-length learnable method refers to the support update of the quantization step when quantizing the transformed image detection model. The feature distillation method refers to the use of the transformed image detection model before quantization to train the quantized transformed image detection model. After the server 102 completes the training of the transformed image detection model, it can send the transformed image detection model to the terminal 101, complete the deployment of the transformed image detection model on the terminal 101, and the terminal 101 calls the transformed image detection model to perform target detection. Optionally, the terminal 101 can be a drone, an autonomous driving vehicle, a mobile phone, etc., which is not limited in the embodiments of the present application.

[0067] Figure 2 This is a flowchart of a training method for an image detection model deployed on a terminal provided by an embodiment of the present application. This embodiment of the present application is illustrated by taking the execution subject as a server as an example. The embodiment includes:

[0068] 201. The server obtains an image detection model to be deployed on the terminal. The image detection model is used to perform target detection on the image, and the image detection model includes multiple branch structures.

[0069] The image detection model is used to perform target detection to detect a target object in an image. For example, the target object includes a car, a bicycle, a tree, a dog, etc., which is not limited in this embodiment of the present application. Optionally, the image detection model is a reparameterized model.

[0070] 202. The server merges multiple branch structures of the image detection model to obtain a modified image detection model including a single branch structure.

[0071] Among them, the branch structure refers to the path by which the image detection model processes the image. The multi-branch structure of the image detection model means that the image detection model uses multiple paths to process the image. Each branch structure includes at least one model layer, and each model layer processes the image to obtain image features. In some embodiments, each branch structure includes a convolution layer, a normalization layer, an activation quantization layer, a weight quantization layer, etc., which is not limited in the embodiments of the present application.

[0072] For example, the image detection model includes two branch structures, namely branch structure 1 and branch structure 2. The model layers included in branch structure 1 from front to back are activation quantization layer, weight quantization layer, 3*3 convolution layer, and normalization layer. The model layers included in branch structure 2 from front to back are activation quantization layer, weight quantization layer, 1*1 convolution layer, and normalization layer.

[0073] In an embodiment of the present application, the image detection model includes multiple branch structures. If the image detection model is trained, multiple branch structures will be trained. In order to simplify the image detection model, the multiple branch structures of the image detection model can be merged to form a modified image detection model with a single branch structure, so that the modified image detection model with a single branch structure can be trained during training, and the training speed can be improved on the basis of simplifying the structure of the image detection model.

[0074] 203. The server trains the transformed image detection model based on the sample image set by using a gradient mask method, and quantizes the trained transformed image detection model to obtain a quantized transformed image detection model.

[0075] Among them, the sample image set includes multiple sample images, and each sample image is used to train the transformed image detection model. For example, each sample image includes a car, a bicycle or a cat, etc., which is not limited in the embodiments of the present application. Quantization refers to the process of approximating the continuous value of a signal to multiple discrete values. It can also be understood that quantization refers to the process of compressing the model to reduce the size of the model. The gradient mask refers to expanding the gradient of the transformed image detection model so that the transformed image detection model can simulate the parameters of the multiple branch structures of the image detection model before the transformation, ensuring that the accuracy of the transformed image detection model and the image detection model remains unchanged.

[0076] In an embodiment of the present application, after the server obtains the sample image set, it can use the gradient mask method to train the transformed image detection model to obtain a trained transformed image detection model, so that the transformed image detection model has the ability to perform target detection. Subsequently, the trained transformed image detection model is quantized to obtain a quantized transformed image detection model.

[0077] 204. When the accuracy of the transformed image detection model after quantization is less than the preset accuracy, the server performs quantization perception training on the trained transformed image detection model based on the sample image set using a step-size learnable method and a feature distillation method to obtain the trained transformed image detection model. The step-size learnable method refers to supporting updates of the quantization step size when quantizing the transformed image detection model, and the feature distillation method refers to using the transformed image detection model before quantization to train the quantized transformed image detection model.

[0078] 205. The server deploys the trained transformed image detection model to the terminal.

[0079] The accuracy of the modified image detection model refers to the accuracy of the modified image detection model when performing target detection. The preset accuracy refers to the accuracy at which the modified image detection model meets the usage conditions. For example, the preset accuracy may be 45%, 48%, or other values, which are not limited in the present embodiment.

[0080] In an embodiment of the present application, if the accuracy of the quantized transformed image detection model is less than the preset accuracy, it means that the accuracy of the quantization of the transformed image detection model is too low at this time, and other methods need to be adopted to further improve the accuracy of the transformed image detection model. It is necessary to further train the trained transformed image detection model using a step-size learnable method and a feature distillation method in order to obtain a transformed image detection model that meets the accuracy requirements and can also be quantized. The server can then deploy the trained transformed image detection model to the terminal, and the terminal will use the transformed image detection model.

[0081] The training method of the image detection model provided in the embodiment of the present application transforms the image detection model with a multi-branch structure into a single-branch structure, and then simultaneously adopts a step-size learnable method and a feature distillation method to perform quantitative perception training on the transformed image detection model to ensure the accuracy of the transformed image detection model after the quantitative perception training. After the transformed image detection model after the quantitative perception training is deployed on the terminal, the structure of the model is quantized so that the terminal can operate the transformed image detection model normally, and the completed training ensures the accuracy requirements of the terminal using the transformed image detection model for detection, and the training process of the image detection model with a single-branch structure is fast, thereby improving the training efficiency.

[0082] Figure 3 This is a flowchart of a training method for an image detection model deployed on a terminal provided by an embodiment of the present application. This embodiment of the present application is illustrated by taking the execution subject as a server as an example. The embodiment includes:

[0083] 301. The server obtains an image detection model to be deployed on the terminal. The image detection model is used to perform target detection on the image, and the image detection model includes multiple branch structures.

[0084] It should be noted that the embodiment of the present application is described using the image detection model as an example. In another embodiment, the image detection model in the embodiment of the present application may also have other functions, such as image segmentation function, or other functions, which are not limited in the present application.

[0085] 302. The server expands a weight matrix corresponding to each branch structure in the plurality of branch structures to obtain an expanded weight matrix of a preset size.

[0086] Among them, the preset size can be 3*3, 4*4, 5*5 or other sizes, which is not limited in the embodiments of the present application. For the acquired image detection model, the image detection model includes multiple branch structures, each branch structure corresponds to a weight matrix, and in order to merge multiple branch structures, the weight matrix of each branch structure needs to be expanded to an extended weight matrix of a preset size, so that the multiple branch structures can be merged based on the extended weight matrix.

[0087] Optionally, expanding the weight matrix of the branch structure includes: adding M columns of all-zero vectors or N rows of all-zero vectors outside the weight matrix of the branch structure, so that the size of the weight matrix after adding N rows and / or M columns of all-zero vectors reaches a preset size, where N and M are both positive integers.

[0088] For example, if the image detection model includes three branch structures, namely branch structure 1, branch structure 2 and branch structure 3, if branch structure 1 is a 1*1 weight matrix, branch structure 2 is a 3*3 weight matrix, and branch structure 3 is a 1*1 weight matrix, and the preset size is 3*3, then branch structure 2 does not need to be expanded, and branch structure 1 and branch structure 3 are expanded to 3*3 weight matrices. After expansion, branch structure 1, branch structure 3 and unexpanded branch structure 2 are all 3*3 weight matrices. For example, if the 1*1 weight matrix is ​​[1], the 1*1 weight matrix is ​​expanded to a 3*3 matrix, then a row of all-zero vectors can be added below, to the left, above and to the right of the 1*1 matrix to obtain a 3*3 matrix.

[0089] 303. The server performs weighted processing on the extended weight matrix corresponding to each branch structure to obtain a transformed image detection model including a single branch structure.

[0090] In an embodiment of the present application, after each branch structure of the image detection model is expanded into an extended branch structure of a preset size, the size of the weight matrix of each branch structure is the same, so the extended weight matrix of each branch structure can be weighted to obtain a modified image detection model including a single branch structure.

[0091] In some embodiments, the server adds the extended weight matrices corresponding to each branch structure to obtain an added weight matrix, and the added weight matrix is ​​the weight matrix corresponding to the single branch structure of the modified image detection model.

[0092] In some embodiments, the image detection model is RepDet (a detection model). The reconstructed image detection model is RepOptDet (a model structure).

[0093] For example, Figure 4 As shown in the figure, the modified image detection model includes a pseudo activation quantization layer and a pseudo weight quantization layer, which are respectively connected to a 3*3 convolutional layer, and the 3*3 convolutional layer is connected to a batch normalization layer. Subsequently, the image detection model directly deploys a 3*3 fusion layer.

[0094] 304. The server trains the transformed image detection model based on the sample image set by using a gradient mask method, and quantizes the trained transformed image detection model to obtain a quantized transformed image detection model.

[0095] In some embodiments, the sample image set includes multiple sample images. For each sample image, the classification label corresponding to the sample image is called, and the sample image is identified using a gradient mask to obtain a predicted label. Then, based on the difference between the classification label and the predicted label, the transformed image detection model is trained to obtain a trained transformed image detection model.

[0096] In some embodiments, the present application uses a gradient mask method to train the transformed image detection model, which is called a reparameterized optimizer technology. That is to say, the use of this reparameterized optimizer can achieve the effect of a single-branch structure with a multi-dimensional extended gradient, ensure the distribution of the weight parameters of the transformed image detection model, and ensure the efficiency of training and quantization of the transformed image detection model.

[0097] In some embodiments, the trained reshaped image detection model can be quantized using uniform quantization and non-uniform quantization. That is, in embodiments of the present application, the trained reshaped image detection model can be quantized using uniform quantization or non-uniform quantization. Optionally, uniform quantization includes binarization or linear quantization. Non-uniform quantization includes logarithmic quantization.

[0098] 305. When the accuracy of the quantized transformed image detection model is less than a preset accuracy, the server obtains a scaling factor of the transformed image detection model, where the scaling factor is used to quantize the transformed image detection model.

[0099] In an embodiment of the present application, quantizing the transformed image detection model will affect the accuracy of the transformed image detection model. Therefore, if the trained transformed image detection model is quantized, it is necessary to first determine whether the accuracy of the quantized transformed image detection model meets the requirements. If it does not meet the requirements, it is necessary to continue to perform quantitative perception training on the trained transformed image detection model so that the accuracy of the trained and quantized transformed image detection model also meets the requirements.

[0100] In some embodiments, after obtaining the quantized transformed image detection model, a test image set is obtained, which includes multiple test images. The quantized transformed image detection model is called to detect each of the multiple test images to obtain a detection label corresponding to each test image. The accuracy of the quantized transformed image detection model is determined based on the classification label and detection label corresponding to each test image in the test image set.

[0101] For example, a first number of test images having the same classification label and detection label is obtained, and a ratio of the first number to a second number of test images included in the test image set is determined as the accuracy of the quantified transformed image detection model.

[0102] In an embodiment of the present application, if it is determined that the accuracy of the transformed image detection model after quantization is less than the preset accuracy, it means that the accuracy of the transformed image detection model after quantization does not meet the requirements. Therefore, the quantization factor when the transformed image detection model is quantized can be obtained, and the quantization factor can be updated to improve the accuracy of the quantization of the transformed image detection model, and subsequently ensure that the accuracy of the transformed image detection model after quantization also meets the requirements.

[0103] In some embodiments, the preset accuracy is 46%, 47%, 50%, or other values, which are not limited in the embodiments of the present application. For example, if the preset accuracy is 46%, and the accuracy of the quantized transformed image detection model is 45%, it means that the accuracy of the quantized transformed image detection model does not meet the requirements and the accuracy of the transformed image detection model needs to be improved.

[0104] In some embodiments, a first accuracy of a trained remodeled image detection model and a second accuracy of a quantized remodeled image detection model are obtained. If the difference between the first accuracy and the second accuracy is greater than an accuracy threshold, it indicates that the accuracy of the quantized remodeled image detection model is less than a preset accuracy. The accuracy threshold can be 0.5%, 1%, 1.5%, or other values, which are not limited in the present embodiments.

[0105] For example, if the first accuracy of the trained transformed image detection model is 46%, the second accuracy of the quantized transformed image detection model is 45%, and the accuracy threshold is 0.5%, it can be determined that the difference between the first accuracy and the second accuracy is greater than the accuracy threshold, and it is determined that the accuracy of the quantized transformed image detection model is less than the preset accuracy.

[0106] 306. The server updates the weights and scaling factors of the transformed image detection model based on the weights and scaling factors of the transformed image detection model.

[0107] The remodeled image detection model includes default weights and scaling factors. After obtaining the scaling factors and weights of the remodeled image detection model, the server can update the default scaling factors and weights based on the scaling factors and weights. When the remodeled image detection model is quantized using the updated weights and scaling factors, the accuracy of the remodeled image detection model will be affected. Therefore, when the accuracy of the quantized remodeled image detection model is less than the preset accuracy, the weights and scaling factors need to be updated to maintain the accuracy when the trained remodeled image detection model is quantized using the updated weights and scaling factors.

[0108] In some embodiments, a first ratio of the activation value and the scaling factor of the transformed image detection model and a second ratio of the weight and the scaling factor are obtained, the first ratio and the second ratio are rounded to integers respectively to obtain a first numerical value corresponding to the first ratio and a second numerical value corresponding to the second ratio, the product of the first numerical value and the scaling factor and the product of the second numerical value and the scaling factor are determined as the pseudo-quantized activation value and weight, the loss value is calculated based on the pseudo-quantized activation value, the pseudo-quantized weight and the scaling factor, the loss value is back-propagated to obtain the updated weight and scaling factor.

[0109] For example, the following formula is used to pseudo-quantize activation values ​​and weights:

[0110]

[0111] in, is the pseudo-quantized activation value or weight, scale is the scaling factor, Round(.) is rounded to the nearest integer, and v is the activation value or weight.

[0112] For example, Figure 5 As shown in the figure, based on the sample image set, the weights and activation values ​​are input respectively. After the weights are quantized and dequantized, the pseudo-quantized weights are obtained. After the activation values ​​are quantized and dequantized, the pseudo-quantized activation values ​​are obtained. The pseudo-quantized activation values ​​and weights are used to perform convolution processing on the sample image set, and then the quantization perception training of the transformed image detection model is completed through the linear correction unit.

[0113] 307. The server detects the sample image set based on the trained transformed image detection model and the updated weights and scaling factors to obtain a prediction result.

[0114] In an embodiment of the present application, the trained transformed image detection model has the ability to perform target detection, and the updated weights and scaling factors will also reduce the impact of quantization on the accuracy of the trained transformed image detection model. Therefore, based on the trained transformed image detection model and the pseudo-quantized activation values ​​and weights, the sample image set is detected to obtain a prediction result. Subsequently, the trained transformed image detection model can continue to be subjected to quantitative perception training based on the prediction result.

[0115] 308. The server trains the trained transformed image detection model based on the difference between the sample result corresponding to the sample image set and the prediction result to obtain the trained transformed image detection model.

[0116] In an embodiment of the present application, each sample image in the sample image set corresponds to a sample result, and there is a difference between the sample result corresponding to the sample image and the predicted result. The parameters of the trained transformed image detection model can be adjusted according to the obtained difference so that the accuracy of the trained transformed image detection model meets the requirements, and the trained transformed image detection model is obtained.

[0117] An embodiment of the present application provides a solution for updating the weights and scaling factors of a transformed image detection model, in which the weights and scaling factors of the transformed image detection model are updated based on the weights and scaling factors of the transformed image detection model, ensuring that when the trained transformed image detection model is quantized using the updated weights and scaling, the accuracy of the quantized transformed image detection model also meets the requirements, ensuring that the transformed image detection model can maintain the same accuracy during target detection while achieving compression.

[0118] 309. Based on the sample image set, the server uses a feature distillation method to perform quantitative perception training on the trained transformed image detection model to obtain the trained transformed image detection model. The feature distillation method refers to using the pre-quantized transformed image detection model to train the quantized transformed image detection model.

[0119] In an embodiment of the present application, the accuracy of the transformed image detection model before quantization is higher than the accuracy of the transformed image detection model after quantization. Therefore, the trained transformed image detection model can be trained with quantization perception by using feature distillation. That is to say, the transformed image detection model before quantization is used to train the transformed image detection model after quantization, so that the accuracy of the transformed image detection model after quantization reaches the accuracy of the transformed image detection model before quantization.

[0120] It should be noted that the above steps 305-308 are a possible implementation of the step size learnable method in the above step 204. In another embodiment, a possible implementation of the feature distillation method in the above step 204 is as follows:

[0121] In some embodiments, the transformed image detection model before quantization is used as a teacher model, and the transformed image detection model after quantization is used as a student model. Based on the teacher model and the student model, target detection is performed on the sample image set to obtain a first feature matrix and a first detection result output by the teacher model, and a second feature matrix and a second detection result output by the student model. Based on the difference between the first feature matrix and the second feature matrix, and the difference between the first detection result and the second detection result, the student model is trained to obtain a trained transformed image detection model.

[0122] For example, Figure 6 As shown, the sample image set is input into the teacher model and the student model respectively, and the sample image set is processed by the backbone network, neck network, feature extraction network and head network of the teacher model. The feature extraction network can extract the first feature matrix, and the head network can extract the first detection result. The sample image set is processed by the backbone network, neck network, feature extraction network and head network of the student model. The feature extraction network can extract the second feature matrix, and the head network can extract the second detection result. Based on the difference between the first feature matrix and the second feature matrix and the difference between the first detection result and the second detection result, the student model is trained to obtain the trained transformed image detection model.

[0123] In the embodiments of the present application, two methods, a step-size learnable method and a feature distillation method, are used to perform quantitative perception training on the trained transformed image detection model. The server can simultaneously use the step-size learnable method and the feature distillation method to perform quantitative perception training, or, the order of using the above two methods can be determined based on the accuracy of the trained transformed image detection model.

[0124] In some embodiments, when the accuracy of the quantized transformed image detection model is less than the preset accuracy, the trained transformed image detection model is subjected to quantitative perceptual training using a step-size learnable method based on the sample image set. When the trained transformed image detection model is subjected to quantitative training using a step-size learnable method and the accuracy of the obtained transformed image detection model is less than the preset accuracy, the transformed image detection model is subjected to quantitative perceptual training using a feature distillation method.

[0125] In some embodiments, multiple model layers of the trained transformed image detection model are obtained. When model layers of the same type exist in the multiple model layers, the model layers of the same type are fused to obtain a fused transformed image detection model. The accuracy of the fused transformed image detection model is the same as that of the trained transformed image detection model.

[0126] In some embodiments, model layers of the same type in multiple model layers can be vertically fused, or horizontally fused. Optionally, model layers of the same type refer to model layers with a linear relationship, or model layers of the same type refer to model layers with the same structure. For example, if model layers of the same type refer to model layers with a linear relationship, then the model layers of the same type can be vertically fused, and if model layers of the same type refer to model layers with the same structure, then the model layers of the same type can be horizontally fused.

[0127] It should be noted that the fusion of multiple model layers in the embodiment of the present application will not change the model structure, and therefore will not affect the accuracy of the model. However, since the fusion of multiple model layers will reduce memory access, the memory consumption of the running model will be reduced, and the access speed will be improved.

[0128] For example, Figure 7 As shown in the figure, if the modified image detection model includes a convolution layer, a batch normalization layer, a linear correction layer, a linear quantization layer, and a linear dequantization layer, then the linear dequantization layer, the convolution layer, the batch normalization layer, the linear correction layer, and the linear quantization layer can be fused to obtain a quantized CBR structure. If the modified image detection model includes a first convolution layer, a second convolution layer, a first activation function layer, and a second activation function layer, then the first convolution layer and the second convolution layer can be fused to obtain a fused convolution layer, and the first activation function layer and the second activation function layer can be fused to obtain an activation function layer.

[0129] In some embodiments, the server can also fuse the transformed image detection model based on the hardware parameters to ensure that the hardware can run successfully when running the fused transformed image detection model, thereby improving the efficiency of running the transformed image detection model. Figure 8 As shown, the transformed image detection model is split into multiple sub-graphs, and the multiple sub-graphs are horizontally and vertically fused. The fusion result that meets the requirements is determined based on the obtained fusion result and the hardware parameters. Based on the fusion result, the corresponding sub-graphs are fused to obtain the fused transformed image detection model.

[0130] It should be noted that the method of fusing the model layers to obtain the fused transformed image detection model in the embodiment of the present application can also be referred to as graph optimization technology. In other words, the model layers of the transformed image detection model are fused using graph optimization technology.

[0131] 310. The server deploys the trained transformed image detection model to the terminal.

[0132] The following diagram illustrates the training process of the image detection model in the embodiment of the present application. Figure 9 , the process includes:

[0133] 1. Obtain the image detection model.

[0134] 2. Perform structural transformation on the image detection model to obtain a transformed image detection model.

[0135] 3. Train the transformed image detection model based on the sample image set to obtain a trained transformed image detection model.

[0136] 4. Quantify the trained transformed image detection model to obtain a quantized transformed image detection model.

[0137] 5. If the accuracy of the quantized transformed image detection model is less than the preset accuracy, execute step 6. If the accuracy of the quantized transformed image detection model is not less than the preset accuracy, perform graph optimization.

[0138] 6. Use a step-size learnable method for quantitative perception training.

[0139] 7. If the accuracy of quantized perception training using a step-size learnable method is less than the preset accuracy, execute step 8. If the accuracy is not less than the preset accuracy, perform graph optimization.

[0140] 8. Use feature distillation to perform quantitative perception training.

[0141] 9. If the accuracy of quantized perception training using feature distillation is less than the preset accuracy, other methods are used for training. If the accuracy is not less than the preset accuracy, graph optimization is performed.

[0142] In the solution provided in the embodiment of the present application, the image detection model with a multi-branch structure is transformed into a single-branch structure, and then the transformed image detection model is subjected to quantitative perception training by using a step-size learnable method and a feature distillation method at the same time to ensure the accuracy of the transformed image detection model after the quantitative perception training. After the transformed image detection model after the quantitative perception training is deployed on the terminal, the structure of the model is quantized so that the terminal can normally operate the transformed image detection model, and the completed training ensures that the terminal uses the transformed image detection model for detection with the accuracy requirements, and the training process of the image detection model with a single-branch structure is fast, thereby improving the training efficiency.

[0143] In addition, the scaling factor of the transformed image detection model is updated based on the activation value and weight of the transformed image detection model to ensure that when the trained transformed image detection model is quantized using the updated scaling factor, the accuracy of the quantized transformed image detection model also meets the requirements, ensuring that the transformed image detection model can maintain the same accuracy during target detection while achieving compression.

[0144] In addition, the trained transformed image detection model is subjected to quantization perception training by using feature distillation. That is, the transformed image detection model before quantization is used to train the quantized transformed image detection model, so that the accuracy of the quantized transformed image detection model reaches the accuracy of the transformed image detection model before quantization.

[0145] Figure 10 This is a structural diagram of a training device for an image detection model deployed on a terminal provided by an embodiment of the present application, see Figure 10 , the device comprises:

[0146] An acquisition module 1001 is configured to acquire an image detection model to be deployed on a terminal, wherein the image detection model is configured to perform target detection on an image and includes multiple branch structures.

[0147] A merging module 1002 is configured to merge the multiple branch structures of the image detection model to obtain a modified image detection model including a single branch structure;

[0148] A training module 1003 is configured to train the reshaped image detection model based on a sample image set using a gradient mask method, and quantize the trained reshaped image detection model to obtain a quantized reshaped image detection model;

[0149] The training module 1003 is further configured to, when the accuracy of the quantized reshaped image detection model is less than a preset accuracy, perform quantization-aware training on the trained reshaped image detection model based on the sample image set using a step-size learnable method and a feature distillation method to obtain a trained reshaped image detection model, wherein the step-size learnable method supports updating the quantization step size when the reshaped image detection model is quantized, and the feature distillation method uses the reshaped image detection model before quantization to train the reshaped image detection model after quantization;

[0150] The deployment module 1004 is used to deploy the trained transformed image detection model to the terminal.

[0151] In a possible implementation, the merging module 1002 is configured to:

[0152] For each branch structure of the plurality of branch structures, expanding a weight matrix corresponding to the branch structure to obtain an expanded weight matrix of a preset size;

[0153] The extended weight matrix corresponding to each branch structure is weighted to obtain a reconstructed image detection model including the single branch structure.

[0154] In a possible implementation, the training module 1003 is configured to:

[0155] Obtaining a scaling factor of the reconstructed image detection model, wherein the scaling factor is used to quantize the reconstructed image detection model;

[0156] Based on the weights of the reconstructed image detection model and the scaling factor, updating the weights and the scaling factor of the reconstructed image detection model;

[0157] Based on the trained transformed image detection model and the updated weights and scaling factors, the sample image set is detected to obtain a prediction result;

[0158] Based on the difference between the sample result corresponding to the sample image set and the prediction result, the trained reconstructed image detection model is trained to obtain the trained reconstructed image detection model.

[0159] In a possible implementation, the training module 1003 is configured to:

[0160] Obtaining a first ratio of an activation value of the transformed image detection model to the scaling factor and a second ratio of the weight to the scaling factor;

[0161] Rounding the first ratio and the second ratio to integers respectively to obtain a first value corresponding to the first ratio and a second value corresponding to the second ratio;

[0162] Determine the product of the first value and the scaling factor and the product of the second value and the scaling factor as the pseudo-quantized activation value and weight;

[0163] Calculate the loss value based on the pseudo-quantized activation value, the pseudo-quantized weight and the scaling factor;

[0164] Backpropagate the loss value to obtain updated weights and scaling factors.

[0165] In a possible implementation, the training module 1003 is configured to:

[0166] The transformed image detection model before quantization is used as the teacher model, and the transformed image detection model after quantization is used as the student model;

[0167] Based on the teacher model and the student model, target detection is performed on the sample image set to obtain a first feature matrix and a first detection result output by the teacher model, and a second feature matrix and a second detection result output by the student model;

[0168] Based on the difference between the first feature matrix and the second feature matrix, and the difference between the first detection result and the second detection result, the student model is trained to obtain a trained transformed image detection model.

[0169] In a possible implementation, the training module 1003 is configured to:

[0170] When the accuracy of the quantized reshaped image detection model is less than a preset accuracy, performing quantization perception training on the trained reshaped image detection model based on the sample image set using the step-size learnable method;

[0171] When the trained transformed image detection model is quantitatively trained using the step-size learnable method and the accuracy of the transformed image detection model obtained is less than the preset accuracy, the transformed image detection model is quantitatively trained using the feature distillation method.

[0172] In a possible implementation, the acquisition module 1001 is used to acquire multiple model layers of the trained transformed image detection model;

[0173] The device also includes: the fusion module, which is used to fuse the model layers of the same type when there are model layers of the same type in the multiple model layers, to obtain a fused transformed image detection model, and the accuracy of the fused transformed image detection model is the same as the accuracy of the transformed image detection model after training.

[0174] In a possible implementation, the model layers of the same type refer to model layers having a linear relationship, or the model layers of the same type refer to model layers having the same structure.

[0175] It should be noted that the image detection model training device provided in the above embodiment only uses the division of the above functional modules as an example when training the model. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal can be divided into different functional modules to complete all or part of the functions described above. In addition, the image detection model training device provided in the above embodiment and the image detection model training method embodiment are of the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0176] Figure 11 FIG1 shows a block diagram of a terminal 1100 provided by an exemplary embodiment of the present application. The terminal 1100 includes a processor 1101 and a memory 1102 .

[0177] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0178] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one program code, which is executed by the processor 1101 to implement the training method of the image detection model for terminal deployment provided in the method embodiment of the present application.

[0179] In some embodiments, terminal 1100 may optionally include a peripheral device interface 1103 and at least one peripheral device. Processor 1101, memory 1102, and peripheral device interface 1103 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1103 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a camera 1106, a positioning component 1108, and a power supply 1109.

[0180] The peripheral device interface 1103 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0181] The camera assembly 1106 is used to capture images or videos. Optionally, the camera assembly 1106 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0182] Positioning component 1108 is used to locate the current geographic location of terminal 1100 to implement navigation or LBS (Location Based Service). Positioning component 1108 can be based on the US GPS (Global Positioning System), China's BeiDou system, Russia's Greninja system, or the European Union's Galileo system.

[0183] Power supply 1109 is used to power various components in terminal 1100. Power supply 1109 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1109 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0184] In some embodiments, the terminal 1100 further includes one or more sensors 1110 , including but not limited to: an acceleration sensor 1111 , a gyroscope sensor 1112 , a pressure sensor 1113 , a fingerprint sensor 1114 , an optical sensor 1115 , and a proximity sensor 1116 .

[0185] The accelerometer 1111 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 1100. For example, the accelerometer 1111 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 1101 can control the display screen 1105 to display the user interface in a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 1111. The accelerometer 1111 can also be used to collect game or user motion data.

[0186] The gyroscope sensor 1112 can detect the orientation and rotation angle of the terminal 1100. The gyroscope sensor 1112 can work with the acceleration sensor 1111 to collect the user's 3D movements on the terminal 1100. Based on the data collected by the gyroscope sensor 1112, the processor 1101 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0187] The pressure sensor 1113 can be set on the side frame of the terminal 1100 and / or the lower layer of the display screen 1105. When the pressure sensor 1113 is set on the side frame of the terminal 1100, it can detect the user's grip signal of the terminal 1100, and the processor 1101 performs left and right hand recognition or shortcut operations based on the grip signal collected by the pressure sensor 1113. When the pressure sensor 1113 is set on the lower layer of the display screen 1105, the processor 1101 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 1105. The operable controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0188] The fingerprint sensor 1114 is used to collect the user's fingerprint. The processor 1101 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 1114, or the fingerprint sensor 1114 identifies the user's identity based on the collected fingerprint. When the user's identity is recognized as a trusted identity, the processor 1101 authorizes the user to perform relevant sensitive operations, such as unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1114 can be set on the front, back, or side of the terminal 1100. When a physical button or manufacturer logo is provided on the terminal 1100, the fingerprint sensor 1114 can be integrated with the physical button or manufacturer logo.

[0189] Optical sensor 1115 is used to collect ambient light intensity. In one embodiment, processor 1101 can control the display brightness of display screen 1105 based on the ambient light intensity collected by optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of display screen 1105 is increased; when the ambient light intensity is low, the display brightness of display screen 1105 is decreased. In another embodiment, processor 1101 can also dynamically adjust the shooting parameters of camera assembly 1106 based on the ambient light intensity collected by optical sensor 1115.

[0190] Proximity sensor 1116, also known as a distance sensor, is disposed on the front panel of terminal 1100. Proximity sensor 1116 is used to detect the distance between the user and the front of terminal 1100. In one embodiment, when proximity sensor 1116 detects that the distance between the user and the front of terminal 1100 is gradually decreasing, processor 1101 controls display screen 1105 to switch from the screen-on state to the screen-off state. When proximity sensor 1116 detects that the distance between the user and the front of terminal 1100 is gradually increasing, processor 1101 controls display screen 1105 to switch from the screen-off state to the screen-on state.

[0191] Those skilled in the art will understand that Figure 11 The structure shown in the figure does not constitute a limitation on the terminal 1100, and the terminal 1100 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0192] Figure 121 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1200 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1201 and one or more memories 1202, wherein the memory 1202 stores at least one program code, and the at least one program code is loaded and executed by the processor 1201 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0193] The server 1200 is used to execute the steps executed by the server in the above method embodiment.

[0194] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including program code. The program code can be executed by a processor in a computer device to implement the training method for the image detection model for terminal deployment in the above embodiment. For example, the computer-readable storage medium can be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0195] In an exemplary embodiment, a computer program or computer program product is also provided, which includes a computer program code. When the computer program code is executed by a server, the server implements the training method of the image detection model for terminal deployment in the above embodiment.

[0196] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments may be accomplished by hardware, or may be accomplished by a program instructing the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0197] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A training method for an image detection model deployed on a terminal, characterized in that: The method comprises: Obtain an image detection model to be deployed on the terminal, wherein the image detection model is used to perform target detection on the image, and the image detection model includes multiple branch structures; Merging the multiple branch structures of the image detection model to obtain a modified image detection model including a single branch structure; Based on the sample image set, the transformed image detection model is trained using a gradient mask method, and the trained transformed image detection model is quantized to obtain a quantized transformed image detection model; When the accuracy of the quantized reshaped image detection model is less than a preset accuracy, based on the sample image set, the trained reshaped image detection model is subjected to quantization-aware training using a step-size learnable method and a feature distillation method to obtain a trained reshaped image detection model, wherein the step-size learnable method refers to supporting updates of the quantization step size when the reshaped image detection model is quantized, and the feature distillation method refers to training the quantized reshaped image detection model using the reshaped image detection model before quantization; The trained remodeled image detection model is deployed on the terminal; and the multiple branch structures of the image detection model are merged to obtain a remodeled image detection model including a single branch structure, including: For each branch structure of the plurality of branch structures, expanding a weight matrix corresponding to the branch structure to obtain an expanded weight matrix of a preset size; The extended weight matrix corresponding to each branch structure is weighted to obtain a reconstructed image detection model including the single branch structure.

2. The method according to claim 1, characterized in that The step of performing quantitative perceptual training on the trained transformed image detection model based on the sample image set using the step-size learnable method includes: Obtaining a scaling factor of the reconstructed image detection model, wherein the scaling factor is used to quantize the reconstructed image detection model; Based on the weights of the reconstructed image detection model and the scaling factor, updating the weights and the scaling factor of the reconstructed image detection model; Based on the trained transformed image detection model and the updated weights and scaling factors, the sample image set is detected to obtain a prediction result; Based on the difference between the sample result corresponding to the sample image set and the prediction result, the trained reconstructed image detection model is trained to obtain the trained reconstructed image detection model.

3. The method according to claim 2, characterized in that The updating of the weight and the scaling factor of the transformed image detection model based on the weight and the scaling factor of the transformed image detection model includes: Obtaining a first ratio of an activation value of the transformed image detection model to the scaling factor and a second ratio of the weight to the scaling factor; Rounding the first ratio and the second ratio to integers respectively to obtain a first value corresponding to the first ratio and a second value corresponding to the second ratio; Determine the product of the first value and the scaling factor and the product of the second value and the scaling factor as the pseudo-quantized activation value and weight; Calculate the loss value based on the pseudo-quantized activation value, the pseudo-quantized weight and the scaling factor; Backpropagate the loss value to obtain updated weights and scaling factors.

4. The method according to claim 1, wherein The step of performing quantitative perceptual training on the trained transformed image detection model using a feature distillation method based on the sample image set includes: The transformed image detection model before quantization is used as the teacher model, and the transformed image detection model after quantization is used as the student model; Based on the teacher model and the student model, target detection is performed on the sample image set to obtain a first feature matrix and a first detection result output by the teacher model, and a second feature matrix and a second detection result output by the student model; Based on the difference between the first feature matrix and the second feature matrix, and the difference between the first detection result and the second detection result, the student model is trained to obtain a trained transformed image detection model.

5. The method according to claim 1, wherein When the accuracy of the quantized reshaped image detection model is less than a preset accuracy, based on the sample image set, the trained reshaped image detection model is subjected to quantization perception training using a step-size learnable method and a feature distillation method, including: When the accuracy of the quantized reshaped image detection model is less than the preset accuracy, performing quantization perception training on the trained reshaped image detection model based on the sample image set using the step-size learnable method; When the trained transformed image detection model is quantitatively trained using the step-size learnable method and the accuracy of the transformed image detection model obtained is less than the preset accuracy, the transformed image detection model is quantitatively trained using the feature distillation method.

6. The method according to claim 1, wherein The method further comprises: Obtaining multiple model layers of the trained transformed image detection model; When there are model layers of the same type in the multiple model layers, the model layers belonging to the same type are fused to obtain a fused transformed image detection model, and the accuracy of the fused transformed image detection model is the same as the accuracy of the transformed image detection model after training.

7. The method according to claim 6, characterized in that The model layers of the same type refer to model layers having a linear relationship, or the model layers of the same type refer to model layers having the same structure.

8. A training device for an image detection model deployed on a terminal, characterized in that: The device comprises: An acquisition module is used to acquire an image detection model to be deployed on the terminal, wherein the image detection model is used to perform target detection on the image, and the image detection model includes multiple branch structures; a merging module, configured to merge the multiple branch structures of the image detection model to obtain a reconstructed image detection model including a single branch structure, wherein the merging of the multiple branch structures of the image detection model to obtain a reconstructed image detection model including a single branch structure comprises: for each branch structure of the multiple branch structures, expanding a weight matrix corresponding to the branch structure to obtain an expanded weight matrix of a preset size; and performing weighted processing on the expanded weight matrix corresponding to each branch structure to obtain a reconstructed image detection model including the single branch structure; A training module, configured to train the reshaped image detection model based on a sample image set using a gradient mask method, and quantize the trained reshaped image detection model to obtain a quantized reshaped image detection model; The training module is further configured to, when the accuracy of the quantized reshaped image detection model is less than a preset accuracy, perform quantization-aware training on the trained reshaped image detection model based on the sample image set using a step-size learnable method and a feature distillation method to obtain a trained reshaped image detection model, wherein the step-size learnable method refers to supporting updates of the quantization step size when quantizing the reshaped image detection model, and the feature distillation method refers to training the quantized reshaped image detection model using the reshaped image detection model before quantization; A deployment module is used to deploy the trained transformed image detection model to the terminal.

9. A server, characterized in that: The server includes one or more processors and one or more memories, and at least one program code is stored in the one or more memories. The at least one program code is loaded and executed by the one or more processors to implement the operations performed by the training method for the image detection model for terminal deployment as described in any one of claims 1 to 7.

Images