Model fine-tuning method and device, electronic equipment and storage medium
By determining the threshold of redundant parameters and setting hyperparameters in the 3D detection model, the problem of redundant parameters affecting the model inference time is solved, thereby improving the model's accuracy and performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-27
- Publication Date
- 2026-03-31
AI Technical Summary
Existing 3D detection models suffer from redundant parameters during training, which affect inference time and contribute nothing to performance, leading to a decline in model performance.
By determining the parameter thresholds between redundant and non-redundant parameters among multiple model parameters in a pre-trained model, dividing the training states of the convolutional kernels, setting corresponding hyperparameters for different states, and replacing the original hyperparameters, model fine-tuning is achieved.
It improved the accuracy of the target detection model, enhanced the model's performance, and reduced inference time.
Smart Images
Figure CN117313839B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a model fine-tuning method, apparatus, electronic device, and storage medium. Background Technology
[0002] To alleviate road traffic congestion and accidents, the demand for autonomous driving technology is increasing daily. 3D object detection is the foundation of autonomous driving perception technology, and the accuracy of the model directly affects subsequent tracking and control performance.
[0003] To reduce training costs and shorten model delivery time, 3D detection model training is generally based on multi-task training. This involves integrating the model with other autonomous driving perception algorithms, such as segmentation and lane marking algorithms, into the same model. Each perception team then fine-tunes the model for each sub-task based on the multi-task model. During fine-tuning, because the large-scale performance of the neural network model's backbone is already sufficient to extract shallow features of the target, the convolutional layer parameters of the backbone are typically fixed, and only a small subset of parameters are trained. This training method achieves faster convergence and model delivery. However, due to redundancy in the model design, the backbone contains many parameters with very small values. These parameters have almost zero impact on the model and contribute nothing to the result during inference. However, these parameters still participate in the convolution calculation, increasing the model's inference time.
[0004] A common solution is to prune these parameters during quantization to reduce the number of parameters and computational cost in the model. However, excessive pruning can easily lead to a decline in model performance. Therefore, the main goal of fine-tuning 3D detection and perception algorithms is to make full use of these redundant parameters with relatively small values to improve model performance. Summary of the Invention
[0005] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this application provides a model fine-tuning method, apparatus, electronic device and storage medium.
[0006] Firstly, this application provides a model fine-tuning method, including:
[0007] Obtain multiple model parameters of a pre-trained model, and determine a parameter threshold among the multiple model parameters as the boundary between redundant and non-redundant parameters;
[0008] The training state of the multiple convolutional kernels is determined based on the parameter threshold and the convolutional kernel parameters of the multiple convolutional kernels in the pre-trained model.
[0009] Set the corresponding first hyperparameter for the convolution kernel in different training states;
[0010] The first hyperparameter is used to replace the corresponding second hyperparameter in the pre-trained model to obtain a finely tuned target detection model for use in environmental perception for autonomous driving.
[0011] Optionally, determining a parameter threshold among the plurality of model parameters as the boundary between redundant and non-redundant parameters includes:
[0012] Obtain the total number of model parameters;
[0013] Obtain a preset percentage of a first parameter and a percentage of a second parameter, wherein the percentage of the first parameter is less than the percentage of the second parameter.
[0014] The quantity of the first parameter is determined based on the percentage of the first parameter quantity and the total quantity.
[0015] The quantity of the second parameter is determined based on the percentage of the second parameter quantity and the total quantity;
[0016] The multiple model parameters are sorted in ascending order to obtain a model parameter sequence;
[0017] A first threshold is determined in the model parameter sequence based on the number of the first parameter;
[0018] A second threshold is determined in the model parameter sequence based on the number of the second parameter.
[0019] The first threshold and the second threshold are determined as the parameter thresholds.
[0020] Optionally, determining the training state of multiple convolutional kernels based on the parameter threshold and the convolutional kernel parameters of multiple convolutional kernels in the pre-trained model includes:
[0021] Obtain multiple convolutional kernels and the convolutional kernel parameters of each convolutional kernel in the pre-trained model;
[0022] Based on the aforementioned parameter threshold, multiple convolution kernel parameters are added to the corresponding convolution kernel parameter set respectively;
[0023] Based on the parameter set of each convolution kernel, the training state of multiple convolution kernels in the pre-trained model is determined.
[0024] Optionally, based on the parameter threshold, multiple convolutional kernel parameters are added to the corresponding convolutional kernel parameter set, including:
[0025] A first threshold and a second threshold are determined based on the parameter thresholds, wherein the first threshold is less than the second threshold;
[0026] For each of the convolution kernel parameters, if the convolution kernel parameter is less than or equal to a first threshold, the convolution kernel parameter is added to the first convolution kernel parameter set;
[0027] If the convolution kernel parameter is greater than the first threshold and less than the second threshold, the convolution kernel parameter is added to the second convolution kernel parameter set;
[0028] If the convolution kernel parameter is greater than the second threshold, the convolution kernel parameter is added to the third convolution kernel parameter set.
[0029] Optionally, based on each set of convolutional kernel parameters, the training state of multiple convolutional kernels in the pre-trained model is determined, including:
[0030] For each convolution kernel, determine the distribution of the kernel parameters in each set of kernel parameters;
[0031] Based on the parameter distribution, the convolutional kernels are added to the corresponding convolutional kernel set, and different convolutional kernel sets correspond to different training states.
[0032] Optionally, for each convolution kernel, the parameter distribution of the kernel parameters in each of the said convolution kernel parameter sets is determined, including:
[0033] For each convolution kernel, the proportion of the number of convolution kernel parameters to be added to the first convolution kernel parameter set to the total number of convolution kernel parameters in that convolution kernel is determined as the first parameter distribution ratio;
[0034] The proportion of the number of kernel parameters to be added to the second kernel parameter set to the total number of kernel parameters in that kernel is determined as the second parameter distribution ratio.
[0035] The proportion of the number of kernel parameters to be added to the third kernel parameter set to the total number of kernel parameters in that kernel is determined as the third parameter distribution ratio.
[0036] The first parameter distribution ratio, the second parameter distribution ratio, and the third parameter distribution ratio are determined as the parameter distribution.
[0037] Optionally, the convolutional kernels are added to corresponding sets of convolutional kernels based on the parameter distribution, with different sets of convolutional kernels corresponding to different training states, including:
[0038] If the distribution ratio of the first parameter is greater than the third threshold, the convolution kernel is added to the first convolution kernel set, and the first convolution kernel set corresponds to the first training state;
[0039] If the distribution ratio of the second parameter is greater than the third threshold, the convolutional kernel is added to the second set of convolutional kernels, and the second set of convolutional kernels corresponds to the second training state;
[0040] If the distribution ratio of the third parameter is greater than the third threshold, the convolution kernel is added to the third convolution kernel set, which corresponds to the third training state.
[0041] Optionally, a first hyperparameter is set for the convolutional kernel in different training states, including:
[0042] The preset learning rate parameters and preset regularization parameters set when obtaining the pre-trained model;
[0043] Obtain the first weight and the second weight corresponding to the convolution kernel for each training state;
[0044] The learning rate corresponding to the convolution kernel of each training state is determined based on the first weight and the preset learning rate parameter, and the learning rate of the optimizer is determined based on the learning rate corresponding to the convolution kernel of each training state.
[0045] The regularization parameters corresponding to the convolution kernels of each training state are determined based on the second weights and the preset regularization parameters, and the regularization parameters of the optimizer are determined based on the regularization parameters corresponding to the convolution kernels of each training state.
[0046] Optionally, the method further includes:
[0047] Obtain a test dataset, which includes multiple frames of LiDAR data;
[0048] The LiDAR data is input into the pre-trained model frame by frame, so that the pre-trained model outputs activation values of multiple feature maps corresponding to the LiDAR data in the current frame;
[0049] If the probability that the activation value of the multiple feature maps corresponding to the lidar data in the current frame is equal to a preset value is greater than a fourth threshold, the activation function of the multiple feature maps corresponding to the lidar data in the current frame will be switched from the first activation function to the second activation function.
[0050] Secondly, this application provides a model fine-tuning device, comprising:
[0051] The first acquisition module is used to acquire multiple model parameters of the pre-trained model and determine a parameter threshold among the multiple model parameters as the boundary between redundant and non-redundant parameters.
[0052] The first determining module is used to determine the training state of multiple convolutional kernels based on the parameter threshold and the convolutional kernel parameters of multiple convolutional kernels in the pre-trained model.
[0053] The setting module is used to set the corresponding first hyperparameters for the convolution kernels in different training states;
[0054] The replacement module is used to replace the corresponding second hyperparameter in the pre-trained model with the first hyperparameter to obtain a finely tuned target detection model for environmental perception in autonomous driving.
[0055] Optionally, the first acquisition module includes:
[0056] The first acquisition unit is used to acquire the total number of model parameters;
[0057] The second acquisition unit is used to acquire a preset first parameter quantity percentage and a second parameter quantity percentage, wherein the first parameter quantity percentage is less than the second parameter quantity percentage;
[0058] The first determining unit is configured to determine the quantity of the first parameter based on the percentage of the first parameter quantity and the total quantity;
[0059] The second determining unit is used to determine the quantity of the second parameter based on the percentage of the second parameter quantity and the total quantity;
[0060] A sorting unit is used to sort multiple model parameters in ascending order to obtain a model parameter sequence;
[0061] The third determining unit is used to determine a first threshold in the model parameter sequence based on the number of the first parameters;
[0062] The fourth determining unit is used to determine a second threshold in the model parameter sequence based on the number of the second parameters;
[0063] The fifth determining unit is used to determine the first threshold and the second threshold as the parameter threshold.
[0064] Optionally, the first determining module includes:
[0065] The second acquisition unit is used to acquire multiple convolutional kernels in the pre-trained model and the convolutional kernel parameters of each convolutional kernel;
[0066] The first addition unit is used to add multiple convolution kernel parameters to the corresponding convolution kernel parameter set based on the parameter threshold.
[0067] The sixth determining unit is used to determine the training state of multiple convolutional kernels in the pre-trained model based on the parameter set of each convolutional kernel.
[0068] Optionally, the first adding unit includes:
[0069] A first determining subunit is configured to determine a first threshold and a second threshold based on the parameter threshold, wherein the first threshold is less than the second threshold;
[0070] The first addition subunit is used to add the convolution kernel parameter to the first convolution kernel parameter set if the convolution kernel parameter is less than or equal to a first threshold for each convolution kernel parameter.
[0071] The second adding subunit is used to add the convolution kernel parameter to the second convolution kernel parameter set if the convolution kernel parameter is greater than the first threshold and less than the second threshold.
[0072] The third addition subunit is used to add the convolution kernel parameter to the third convolution kernel parameter set if the convolution kernel parameter is greater than the second threshold.
[0073] Optionally, the sixth determining unit includes:
[0074] The second determining subunit is used to determine the parameter distribution of the convolution kernel parameters in each set of convolution kernel parameters for each convolution kernel.
[0075] The fourth addition subunit is used to add the convolutional kernel to the corresponding convolutional kernel set based on the parameter distribution. Different convolutional kernel sets correspond to different training states.
[0076] Optionally, the second determining subunit is further configured to:
[0077] For each convolution kernel, the proportion of the number of convolution kernel parameters to be added to the first convolution kernel parameter set to the total number of convolution kernel parameters in that convolution kernel is determined as the first parameter distribution ratio;
[0078] The proportion of the number of kernel parameters to be added to the second kernel parameter set to the total number of kernel parameters in that kernel is determined as the second parameter distribution ratio.
[0079] The proportion of the number of kernel parameters to be added to the third kernel parameter set to the total number of kernel parameters in that kernel is determined as the third parameter distribution ratio.
[0080] The first parameter distribution ratio, the second parameter distribution ratio, and the third parameter distribution ratio are determined as the parameter distribution.
[0081] Optionally, the fourth adding subunit is further configured to:
[0082] If the distribution ratio of the first parameter is greater than the third threshold, the convolution kernel is added to the first convolution kernel set, and the first convolution kernel set corresponds to the first training state;
[0083] If the distribution ratio of the second parameter is greater than the third threshold, the convolutional kernel is added to the second set of convolutional kernels, and the second set of convolutional kernels corresponds to the second training state;
[0084] If the distribution ratio of the third parameter is greater than the third threshold, the convolution kernel is added to the third convolution kernel set, which corresponds to the third training state.
[0085] Optionally, the setting module includes:
[0086] The third acquisition unit is used to acquire the preset learning rate parameters and preset regularization parameters set during the pre-training of the model;
[0087] The fourth acquisition unit is used to acquire the first weight and the second weight corresponding to the convolution kernel of each training state.
[0088] The seventh determining unit is used to determine the learning rate corresponding to the convolution kernel of each training state based on the first weight and the preset learning rate parameter, and to determine the learning rate of the optimizer based on the learning rate corresponding to the convolution kernel of each training state.
[0089] The eighth determining unit is used to determine the regularization parameter corresponding to the convolution kernel of each training state based on the second weight and the preset regularization parameter, and to determine the regularization parameter of the optimizer based on the regularization parameter corresponding to the convolution kernel of each training state.
[0090] Optionally, the device further includes:
[0091] The second acquisition module is used to acquire a test dataset, which includes multiple frames of LiDAR data.
[0092] The input module is used to input the LiDAR data frame by frame into the pre-trained model so that the pre-trained model outputs the activation values of multiple feature maps corresponding to the LiDAR data in the current frame;
[0093] The switching module is used to switch the activation function of the multiple feature maps corresponding to the lidar data in the current frame from the first activation function to the second activation function if the probability that the activation value of the multiple feature maps corresponding to the lidar data in the current frame is equal to a preset value is greater than a fourth threshold.
[0094] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;
[0095] Memory, used to store computer programs;
[0096] When a processor executes a program stored in memory, it implements the model fine-tuning method described in any of the first aspects.
[0097] Fourthly, this application provides a computer-readable storage medium storing a program for a model fine-tuning method, wherein when the program for the model fine-tuning method is executed by a processor, it implements the steps of the model fine-tuning method described in any of the first aspects.
[0098] The beneficial effects of this invention are:
[0099] This application embodiment determines a parameter threshold among multiple model parameters of a pre-trained model, which serves as the boundary between redundant and non-redundant parameters. This preserves and fully utilizes the features of redundant parameters. Based on the parameter threshold, the training states of multiple convolutional kernels in the pre-trained model are determined, and a corresponding first hyperparameter is set for each convolutional kernel in each training state. The original second hyperparameter is then replaced with the first hyperparameter, thereby achieving fine-tuning of the pre-trained model and obtaining a fine-tuned target detection model. By fully utilizing the features of redundant parameters to fine-tune the pre-trained model, the accuracy of the target detection model can be improved, and the model's performance can be enhanced. Attached Figure Description
[0100] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0101] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0102] Figure 1 A flowchart illustrating a model fine-tuning method provided in this application embodiment;
[0103] Figure 2 A flowchart of step S101 provided in an embodiment of this application;
[0104] Figure 3 A flowchart of step S102 provided in the embodiments of this application;
[0105] Figure 4 A flowchart of step S303 provided in the embodiments of this application;
[0106] Figure 5 A flowchart of step S103 provided in the embodiments of this application;
[0107] Figure 6Another flowchart of a model fine-tuning method provided in an embodiment of this application;
[0108] Figure 7 A structural diagram of a model fine-tuning device is provided in an embodiment of this application;
[0109] Figure 8 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0110] To fully utilize the redundant parameters with relatively small values and improve model performance, thus achieving the main goal of fine-tuning 3D detection and perception algorithms, this application provides a model fine-tuning method, apparatus, electronic device, and storage medium.
[0111] In the embodiments of this application, such as Figure 1 As shown, a model fine-tuning method is provided, including:
[0112] Step S101: Obtain multiple model parameters of the pre-trained model, and determine a parameter threshold among the multiple model parameters as the boundary between redundant and non-redundant parameters;
[0113] In this embodiment of the application, the pre-trained model can refer to the model obtained after pre-training the initial model. The pre-trained model includes: at least one model parameter, multiple convolutional kernels, etc.; wherein, each model parameter can be represented by w. i This means that the total number of model parameters is s = w1 + w 2 +,...,+w n All convolutional kernels can be represented by a set C = (conv1, conv2, ..., convn), and the kernels can be 1x1, 3x3, 5x5, or larger. The parameters of each kernel are (wi1, wi2, ..., win). m ), where wi1 represents the first parameter of the convi-th convolution.
[0114] When fine-tuning pre-trained model parameters, the parameters of shallow convolutional kernels are typically fixed. The basic theory is that this convolution has a strong ability to extract shallow features from the model. Consequently, many model parameter values are relatively large. However, testing has revealed that a certain number of parameter values are still relatively small, i.e., redundant parameters. Therefore, in order to extract features from redundant parameters, a parameter threshold can be determined from among the multiple model parameters to serve as the boundary between redundant and non-redundant parameters. That is, a model parameter is selected from among the multiple model parameters to serve as a parameter threshold. This parameter threshold acts as the boundary between redundant and non-redundant parameters. Model parameters less than or equal to this parameter threshold are considered redundant parameters, and model parameters greater than this parameter threshold are considered non-redundant parameters.
[0115] Step S102: Determine the training state of multiple convolutional kernels based on the parameter threshold and the convolutional kernel parameters of multiple convolutional kernels in the pre-trained model;
[0116] In this embodiment, a single convolutional kernel is composed of multiple kernel parameters. During model training, all parameters of the convolutional kernel are set to the same state, meaning they can only be updated or not updated. For the current convolutional kernel, the parameter distribution of the kernel parameters can be determined according to whether the parameters are updated or not, thus determining the training state during kernel fine-tuning.
[0117] In this step, the convolution kernel parameters of multiple convolution kernels can be divided into different convolution kernel parameter sets according to the parameter threshold, and then the training state of the multiple convolution kernels can be determined according to the parameter distribution of the convolution kernel parameters in each convolution kernel parameter set.
[0118] Step S103: Set the corresponding first hyperparameter for the convolution kernel in different training states;
[0119] After determining the training states of the multiple convolutional kernels, it can be determined which convolutional kernels need to be focused on learning and which convolutional kernels do not need to be focused on learning. Therefore, corresponding first hyperparameters can be set for the convolutional kernels in different training states.
[0120] In this step, the corresponding first hyperparameter can be set for the optimizer corresponding to the convolution kernel in different training states.
[0121] Step S104: Replace the corresponding second hyperparameter in the pre-trained model with the first hyperparameter to obtain a finely tuned target detection model for use in environmental perception for autonomous driving.
[0122] Since the first hyperparameter is obtained based on the relatively small parameter values, it can be used for model fine-tuning. This involves replacing the original second hyperparameter in the pre-trained model with the first hyperparameter, resulting in a finely tuned object detection model. In practical applications, this finely tuned object detection model can be used for environmental perception during autonomous driving.
[0123] This application embodiment determines a parameter threshold among multiple model parameters of a pre-trained model, which serves as the boundary between redundant and non-redundant parameters. This preserves and fully utilizes the features of redundant parameters. Based on the parameter threshold, the training states of multiple convolutional kernels in the pre-trained model are determined, and a corresponding first hyperparameter is set for each convolutional kernel in each training state. The original second hyperparameter is then replaced with the first hyperparameter, thereby achieving fine-tuning of the pre-trained model and obtaining a fine-tuned target detection model. By fully utilizing the features of redundant parameters to fine-tune the pre-trained model, the accuracy of the target detection model can be improved, and the model's performance can be enhanced.
[0124] In yet another embodiment of this application, as Figure 2 As shown, step S101 determines a parameter threshold among the multiple model parameters as the boundary between redundant and non-redundant parameters, including:
[0125] Step S201: Obtain the total number of model parameters;
[0126] Based on the foregoing embodiments, the total number of model parameters s can be obtained.
[0127] Step S202: Obtain a preset first parameter percentage and a second parameter percentage, wherein the first parameter percentage is less than the second parameter percentage;
[0128] The first parameter percentage and the second parameter percentage are preset empirical values, represented by α and β respectively. For example, the first parameter percentage α can be 10% and the second parameter percentage β can be 15%.
[0129] Step S203: Determine the number of the first parameter based on the percentage of the first parameter and the total number;
[0130] In this step, the percentage of the first parameter quantity can be multiplied by the total quantity to obtain the number of the first parameter l1, that is, l1 = α * s. The number of the first parameter is the number of parameter values with smaller values in the first group. For example, the total quantity s is 100, the percentage of the first parameter quantity α is 10%, and the number of the first parameter l1 is 10.
[0131] Step S204: Determine the quantity of the second parameter based on the percentage of the second parameter quantity and the total quantity;
[0132] In this step, the percentage of the second parameter quantity can be multiplied by the total quantity to obtain the number of the second parameter l2, that is, l1 = β*s. The number of the second parameter is the number of parameter values with smaller values in the second group. For example, the total quantity s is 100, the percentage of the second parameter quantity α is 15%, and the number of the second parameter l1 is 15.
[0133] Step S205: Sort the multiple model parameters in ascending order to obtain a model parameter sequence;
[0134] Step S206: Determine a first threshold in the model parameter sequence based on the first parameter quantity;
[0135] In this step, the model parameter corresponding to the first parameter quantity in the model parameter sequence can be directly determined as the first threshold th1. For example, if the first parameter quantity l1 is 10, and the 10th model parameter in the model parameter sequence is p, then p is the first threshold.
[0136] Step S207: Determine a second threshold in the model parameter sequence based on the number of the second parameter;
[0137] In this step, the model parameter corresponding to the number of the second parameter in the model parameter sequence can be directly determined as the second threshold th2. For example, if the number of the second parameter l2 is 15, and the 15th model parameter in the model parameter sequence is q, then q is the second threshold.
[0138] Step S208: Determine the first threshold and the second threshold as the parameter threshold.
[0139] The embodiments of this application can automatically determine the model parameters that serve as the boundary between smaller and larger parameter values among multiple model parameters, and use them as parameter thresholds. This is equivalent to preserving the characteristics of smaller parameter values and making full use of these smaller parameter values so that they can be used for subsequent model fine-tuning.
[0140] In yet another embodiment of this application, as Figure 3 As shown, step S102 determines the training state of multiple convolutional kernels based on the parameter threshold and the convolutional kernel parameters of multiple convolutional kernels in the pre-trained model, including:
[0141] Step S301: Obtain multiple convolutional kernels and the convolutional kernel parameters of each convolutional kernel in the pre-trained model;
[0142] Step S302: Based on the parameter threshold, add multiple convolution kernel parameters to the corresponding convolution kernel parameter set respectively;
[0143] In this step, the size relationship between multiple convolution kernel parameters and parameter thresholds can be determined, and then the multiple convolution kernel parameters are added to the corresponding convolution kernel parameter sets according to the size relationship. The size relationship between the convolution kernel parameters and parameter thresholds in each convolution kernel parameter set is the same.
[0144] In one embodiment of this application, step S302, based on the parameter threshold, adds multiple convolution kernel parameters to the corresponding convolution kernel parameter set, including:
[0145] 1. Determine a first threshold and a second threshold based on the aforementioned parameter thresholds, wherein the first threshold is less than the second threshold;
[0146] Referring to the foregoing embodiments, the parameter thresholds include a first threshold and a second threshold. Since the percentage of the first parameter quantity is less than the percentage of the second parameter quantity, the number of the first parameter is less than the number of the second parameter, so it can be determined that the first threshold is less than the second threshold.
[0147] 2. For each of the convolution kernel parameters, if the convolution kernel parameter is less than or equal to a first threshold, the convolution kernel parameter is added to a first convolution kernel parameter set; if the convolution kernel parameter is greater than the first threshold and less than a second threshold, the convolution kernel parameter is added to a second convolution kernel parameter set; if the convolution kernel parameter is greater than the second threshold, the convolution kernel parameter is added to a third convolution kernel parameter set.
[0148] A first convolutional kernel parameter set s1, a second convolutional kernel parameter set s2, and a third convolutional kernel parameter set s3 can be pre-created. The first, second, and third convolutional kernel parameter sets are used to store convolutional kernel parameters in different states. Specifically, the first convolutional kernel parameter set is used to store convolutional kernel parameters that are less than or equal to a first threshold, the second convolutional kernel parameter set is used to store convolutional kernel parameters that are greater than the first threshold and less than the second threshold, and the third convolutional kernel parameter set is used to store convolutional kernel parameters that are greater than the second threshold.
[0149] For example, for any convolution kernel conv i Its convolution kernel parameters are (w i1 ,w i2 ,...,w im If the convolution kernel parameter w ij If it is less than th1, then w ij Place it into s1; if the parameter w in the convolution kernel ij If it is less than th2 and greater than th1, then w will be... ij Place it in s2; for other states, place it in s3.
[0150] Step S303: Based on each set of convolutional kernel parameters, determine the training state of multiple convolutional kernels in the pre-trained model.
[0151] In this step, the parameter distribution of each convolution kernel parameter in each convolution kernel can be determined, thereby determining the training state of each convolution kernel in the pre-trained model.
[0152] In one embodiment of this application, such as Figure 4 As shown, step S303 determines the training state of multiple convolutional kernels in the pre-trained model based on each set of convolutional kernel parameters, including:
[0153] Step S401: For each convolution kernel, determine the parameter distribution of the convolution kernel parameters in each set of convolution kernel parameters; that is, for each convolution kernel, calculate the parameter distribution ratio of the convolution kernel parameters located in each set of convolution kernel parameters.
[0154] In another embodiment of this application, step S401, for each convolution kernel, determines the parameter distribution of the convolution kernel parameters in each set of convolution kernel parameters, including:
[0155] 1. For each convolution kernel, the proportion of the number of convolution kernel parameters to be added to the first convolution kernel parameter set to the total number of convolution kernel parameters in that convolution kernel is determined as the first parameter distribution ratio;
[0156] 2. The proportion of the number of convolution kernel parameters to be added to the second convolution kernel parameter set to the total number of convolution kernel parameters in that convolution kernel is determined as the second parameter distribution ratio; the proportion of the number of convolution kernel parameters to be added to the third convolution kernel parameter set to the total number of convolution kernel parameters in that convolution kernel is determined as the third parameter distribution ratio;
[0157] Third, the first parameter distribution ratio, the second parameter distribution ratio, and the third parameter distribution ratio are determined as the parameter distribution.
[0158] Based on the foregoing embodiments, for example, the following formula can be used to calculate the sum of any convolutional kernel (conv). i Medium parameter distribution:
[0159]
[0160]
[0161]
[0162] Where p1 represents the convolution kernel conv i In the convolution kernel parameters, the proportion of the number of convolution kernel parameters added to the first convolution kernel parameter set s1 to the total number m of all convolution kernel parameters in that convolution kernel; p2 represents the convolution kernel conv i In the convolution kernel parameters, p3 represents the proportion of the number of convolution kernel parameters added to the second convolution kernel parameter set s2 to the total number m of all convolution kernel parameters in that convolution kernel. iThe proportion of the number of convolution kernel parameters added to the third convolution kernel parameter set s3 to the total number m of all convolution kernel parameters in that convolution kernel.
[0163] Step S402: Based on the parameter distribution, the convolution kernel is added to the corresponding convolution kernel set, and different convolution kernel sets correspond to different training states.
[0164] In another embodiment of this application, step S402 adds the convolutional kernel to the corresponding convolutional kernel set based on the parameter distribution, wherein different convolutional kernel sets correspond to different training states, including:
[0165] If the distribution ratio of the first parameter is greater than the third threshold, the convolutional kernel is added to the first convolutional kernel set, which corresponds to the first training state; if the distribution ratio of the second parameter is greater than the third threshold, the convolutional kernel is added to the second convolutional kernel set, which corresponds to the second training state; if the distribution ratio of the third parameter is greater than the third threshold, the convolutional kernel is added to the third convolutional kernel set, which corresponds to the third training state.
[0166] A first convolutional kernel set S1, a second convolutional kernel set S2, and a third convolutional kernel set S3 can be pre-created. The first, second, and third convolutional kernel sets are used to store convolutional kernels in different states. Specifically, the first convolutional kernel set is used to store the number of convolutional kernels corresponding to the case where the distribution ratio of the first parameter is greater than the third threshold. The second convolutional kernel parameter set is used to store the convolutional kernels corresponding to the case where the distribution ratio of the second parameter is greater than the third threshold. The third convolutional kernel parameter set is used to store the convolutional kernels corresponding to the case where the distribution ratio of the third parameter is greater than the third threshold.
[0167] Based on the foregoing embodiments, for example, if p1 is greater than γ, then the parameter conv is... i If p2 is greater than γ, then the parameter conv is placed in S1. i If p3 is greater than γ, then the parameter conv is placed in S2. i Place it in S3.
[0168] This application embodiment can automatically add the convolution kernel parameters of multiple convolution kernels to different convolution kernel parameter sets according to parameter thresholds. Then, for each convolution kernel, it counts the number of convolution kernel parameters in each convolution kernel parameter set relative to the total number of convolution kernel parameters of that convolution kernel. Based on the parameter distribution of each convolution kernel, the training state of each convolution kernel is obtained. This application embodiment fully considers the parameter distribution of each convolution kernel parameter in each convolution kernel and determines the training state of each convolution kernel based on the parameter distribution, thereby achieving accurate determination of the training state of the convolution kernel.
[0169] In yet another embodiment of this application, as Figure 5 As shown, step S103 sets the corresponding first hyperparameters for the convolutional kernels in different training states, including:
[0170] Step S501: Obtain the preset learning rate parameters and preset regularization parameters set during the pre-training of the model;
[0171] In this embodiment of the application, when pre-training the model, a preset learning rate parameter and a preset regularization parameter can be preset for each optimizer. For example, the preset learning rate parameter is represented by lr and the preset regularization parameter is represented by w.
[0172] Step S502: Obtain the first weight and the second weight corresponding to the convolution kernel of each training state.
[0173] For example, the first weight corresponding to the convolution kernel in the first training state is 1, the first weight corresponding to the convolution kernel in the second training state is 1 / 5, and the first weight corresponding to the convolution kernel in the third training state is 0; the second weight corresponding to the convolution kernel in the first training state is 1 / 5, the second weight corresponding to the convolution kernel in the second training state is 1 / 2, and the second weight corresponding to the convolution kernel in the third training state is 0.
[0174] Step S503: Determine the learning rate corresponding to the convolution kernel of each training state based on the first weight and the preset learning rate parameter, and determine the learning rate of the optimizer based on the learning rate corresponding to the convolution kernel of each training state.
[0175] For example, the learning rate for the convolutional kernel in the first training state is lr1, the learning rate for the convolutional kernel in the second training state is lr2, and the learning rate for the convolutional kernel in the third training state is lr3. The learning rate for each training state's convolutional kernel can be calculated using the following formula:
[0176] lr1=lr
[0177] lr2=lr / 5
[0178] lr3=0
[0179] The learner rate of the optimizer is: optimizer = (s1:lr1, s2:lr2, s3:lr3), which means that for any convolution kernel, the number of convolution kernel parameters located in different convolution kernel parameter sets is divided by the corresponding learner rate.
[0180] Step S504: Determine the regularization parameter corresponding to the convolution kernel of each training state based on the second weight and the preset regularization parameter, and determine the regularization parameter of the optimizer based on the regularization parameter corresponding to the convolution kernel of each training state.
[0181] For example, the regularization parameter corresponding to the convolution kernel in the first training state is w1, the regularization parameter corresponding to the convolution kernel in the second training state is w2, and the regularization parameter corresponding to the convolution kernel in the third training state is w3. The regularization parameter corresponding to the convolution kernel in each training state can be calculated with reference to the following formula:
[0182] lr1=lr
[0183] lr2=lr / 5
[0184] lr3=0
[0185] The embodiments of this application calculate the regularization parameter corresponding to the convolution kernel of each training state based on the second weight corresponding to the convolution kernel of each training state. During fine-tuning, smaller regularization parameters can be designed for the parameters that need to be learned in detail, so as to learn them fully and improve the accuracy of the model.
[0186] The optimizer's regularization parameter is: weight_decay = (s1:w1, s2:w2, s3:w3), which means that for any convolution kernel, the number of convolution kernel parameters located in different convolution kernel parameter sets is divided by the corresponding regularization parameter.
[0187] The embodiments of this application can determine the corresponding learning rate and regularization parameters for the convolution kernel of each training state, and then determine the learning rate and regularization parameters of the optimizer to optimize the accuracy of the model.
[0188] The ReLU activation function is a commonly used choice for model training due to its low complexity and lack of exponential calculation. However, if the activation value is less than 0 during training, the gradient is zero, and there will be no activation on the input data, easily leading to training failure and neuron death. Therefore, the activation function for such neurons should be adjusted. Thus, in another embodiment of this application, during fine-tuning, the distribution of activation values in the feature maps of the model can be statistically analyzed to determine the choice of activation function.
[0189] For the input feature map f i Given C1*H1*W1, output feature map f jThe kernel size is C2*H2*W2, and the convolution parameters are C1*C2*K1*K2, where C1 is the number of input feature channels, H1*W1 is the input feature space size, C2 is the number of output feature channels, H2*W2 is the output feature space size, and K1*K2 is the convolution kernel size. The feature map calculation process is as follows:
[0190] f j =activation(bn(conv(f i )))
[0191] Where conv represents convolution, bn represents batch norm, and activation represents the activation function, typically ReLU. The output feature map f j The state is determined by the activation function, f j The higher the proportion of eigenvalues with a value of 0, the more the current activation function should be modified to mitigate neuronal death. Furthermore, such as... Figure 6 As shown, the model fine-tuning method further includes:
[0192] Step S601: Obtain a test dataset, which includes multiple frames of LiDAR data;
[0193] Step S602: Input the LiDAR data frame by frame into the pre-trained model so that the pre-trained model outputs activation values of multiple feature maps corresponding to the LiDAR data in the current frame;
[0194] Step S603: If the probability that the activation value of the multiple feature maps corresponding to the lidar data in the current frame is equal to a preset value is greater than the fourth threshold, the activation function of the multiple feature maps corresponding to the lidar data in the current frame is switched from the first activation function to the second activation function.
[0195] For example, to determine the activation function during fine-tuning, 100 frames of LiDAR data are randomly selected from the test dataset. These 100 frames of LiDAR data are input into the pre-trained model, and the activation values of all feature maps (f1, f2, ..., fn) are recorded. The current frame feature map f... j All activation values can be represented as f 1j ,f ij ,...,f 100j Where j represents the feature of the j-th frame of LiDAR data in the model, and i represents the feature map of the i-th layer.
[0196] If f 1j ,f 2j ,...,f 100j If the percentage of activation values of 0 in the current feature map is greater than λ, then the activation function of the current feature map is set to the PReLU function.
[0197] This application embodiment analyzes the activation layer based on multiple frames of randomly selected LiDAR data from the test dataset. Different activation functions are designed for different feature layers. Since the performance of the PReLU function is better than that of the ReLU function, the activation function of the model can be optimized, thereby achieving the goal of rapid model delivery.
[0198] In yet another embodiment of this application, as Figure 7 As shown, a model fine-tuning device is also provided, comprising:
[0199] The first acquisition module 11 is used to acquire multiple model parameters of the pre-trained model and determine a parameter threshold among the multiple model parameters as the boundary between redundant and non-redundant parameters.
[0200] The first determining module 12 is used to determine the training state of multiple convolutional kernels based on the parameter threshold and the convolutional kernel parameters of multiple convolutional kernels in the pre-trained model.
[0201] Setting module 13 is used to set the corresponding first hyperparameter for the convolution kernel in different training states;
[0202] Replacement module 14 is used to replace the corresponding second hyperparameter in the pre-trained model with the first hyperparameter to obtain a fine-tuned target detection model for use in environmental perception for autonomous driving.
[0203] Optionally, the first acquisition module includes:
[0204] The first acquisition unit is used to acquire the total number of model parameters;
[0205] The second acquisition unit is used to acquire a preset first parameter quantity percentage and a second parameter quantity percentage, wherein the first parameter quantity percentage is less than the second parameter quantity percentage;
[0206] The first determining unit is configured to determine the quantity of the first parameter based on the percentage of the first parameter quantity and the total quantity;
[0207] The second determining unit is used to determine the quantity of the second parameter based on the percentage of the second parameter quantity and the total quantity;
[0208] A sorting unit is used to sort multiple model parameters in ascending order to obtain a model parameter sequence;
[0209] The third determining unit is used to determine a first threshold in the model parameter sequence based on the number of the first parameters;
[0210] The fourth determining unit is used to determine a second threshold in the model parameter sequence based on the number of the second parameters;
[0211] The fifth determining unit is used to determine the first threshold and the second threshold as the parameter threshold.
[0212] Optionally, the first determining module includes:
[0213] The second acquisition unit is used to acquire multiple convolutional kernels in the pre-trained model and the convolutional kernel parameters of each convolutional kernel;
[0214] The first addition unit is used to add multiple convolution kernel parameters to the corresponding convolution kernel parameter set based on the parameter threshold.
[0215] The sixth determining unit is used to determine the training state of multiple convolutional kernels in the pre-trained model based on the parameter set of each convolutional kernel.
[0216] Optionally, the first adding unit includes:
[0217] A first determining subunit is configured to determine a first threshold and a second threshold based on the parameter threshold, wherein the first threshold is less than the second threshold;
[0218] The first addition subunit is used to add the convolution kernel parameter to the first convolution kernel parameter set if the convolution kernel parameter is less than or equal to a first threshold for each convolution kernel parameter.
[0219] The second adding subunit is used to add the convolution kernel parameter to the second convolution kernel parameter set if the convolution kernel parameter is greater than the first threshold and less than the second threshold.
[0220] The third addition subunit is used to add the convolution kernel parameter to the third convolution kernel parameter set if the convolution kernel parameter is greater than the second threshold.
[0221] Optionally, the sixth determining unit includes:
[0222] The second determining subunit is used to determine the parameter distribution of the convolution kernel parameters in each set of convolution kernel parameters for each convolution kernel.
[0223] The fourth addition subunit is used to add the convolutional kernel to the corresponding convolutional kernel set based on the parameter distribution. Different convolutional kernel sets correspond to different training states.
[0224] Optionally, the second determining subunit is further configured to:
[0225] For each convolution kernel, the proportion of the number of convolution kernel parameters to be added to the first convolution kernel parameter set to the total number of convolution kernel parameters in that convolution kernel is determined as the first parameter distribution ratio;
[0226] The proportion of the number of kernel parameters to be added to the second kernel parameter set to the total number of kernel parameters in that kernel is determined as the second parameter distribution ratio.
[0227] The proportion of the number of kernel parameters to be added to the third kernel parameter set to the total number of kernel parameters in that kernel is determined as the third parameter distribution ratio.
[0228] The first parameter distribution ratio, the second parameter distribution ratio, and the third parameter distribution ratio are determined as the parameter distribution.
[0229] Optionally, the fourth adding subunit is further configured to:
[0230] If the distribution ratio of the first parameter is greater than the third threshold, the convolution kernel is added to the first convolution kernel set, and the first convolution kernel set corresponds to the first training state;
[0231] If the distribution ratio of the second parameter is greater than the third threshold, the convolutional kernel is added to the second set of convolutional kernels, and the second set of convolutional kernels corresponds to the second training state;
[0232] If the distribution ratio of the third parameter is greater than the third threshold, the convolution kernel is added to the third convolution kernel set, which corresponds to the third training state.
[0233] Optionally, the setting module includes:
[0234] The third acquisition unit is used to acquire the preset learning rate parameters and preset regularization parameters set during the pre-training of the model;
[0235] The fourth acquisition unit is used to acquire the first weight and the second weight corresponding to the convolution kernel of each training state.
[0236] The seventh determining unit is used to determine the learning rate corresponding to the convolution kernel of each training state based on the first weight and the preset learning rate parameter, and to determine the learning rate of the optimizer based on the learning rate corresponding to the convolution kernel of each training state.
[0237] The eighth determining unit is used to determine the regularization parameter corresponding to the convolution kernel of each training state based on the second weight and the preset regularization parameter, and to determine the regularization parameter of the optimizer based on the regularization parameter corresponding to the convolution kernel of each training state.
[0238] Optionally, the device further includes:
[0239] The second acquisition module is used to acquire a test dataset, which includes multiple frames of LiDAR data.
[0240] The input module is used to input the LiDAR data frame by frame into the pre-trained model so that the pre-trained model outputs the activation values of multiple feature maps corresponding to the LiDAR data in the current frame;
[0241] The switching module is used to switch the activation function of the multiple feature maps corresponding to the lidar data in the current frame from the first activation function to the second activation function if the probability that the activation value of the multiple feature maps corresponding to the lidar data in the current frame is equal to a preset value is greater than a fourth threshold.
[0242] In another embodiment of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.
[0243] Memory, used to store computer programs;
[0244] When a processor executes a program stored in memory, it implements the model fine-tuning method described in any of the foregoing method embodiments.
[0245] The electronic device provided in this invention allows the processor to execute a program stored in memory to determine a parameter threshold, which serves as the boundary between redundant and non-redundant parameters, among multiple model parameters of a pre-trained model. This preserves and fully utilizes the features of redundant parameters. Based on the parameter threshold, the processor determines the training state of multiple convolutional kernels of the pre-trained model and sets a corresponding first hyperparameter for each training state's convolutional kernel. The first hyperparameter is then used to replace the original second hyperparameter, thereby fine-tuning the pre-trained model and obtaining a fine-tuned target detection model. By fully utilizing the features of redundant parameters to fine-tune the pre-trained model, the accuracy of the target detection model can be improved, and the model's performance can be enhanced.
[0246] The communication bus 1140 mentioned in the above-mentioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 1140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0247] The communication interface 1120 is used for communication between the above-mentioned electronic device and other devices.
[0248] The memory 1130 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0249] The processor 1110 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0250] In another embodiment of this application, a computer-readable storage medium is also provided, on which a program for a model fine-tuning method is stored. When the program for the model fine-tuning method is executed by a processor, it implements the steps of the model fine-tuning method described in any of the foregoing method embodiments.
[0251] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0252] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A model fine-tuning method, characterized in that, The method comprises the following steps: obtaining a plurality of model parameters of a pre-trained model, and determining a parameter threshold value as a boundary between redundant parameters and non-redundant parameters in the plurality of model parameters; determining a parameter threshold value as a boundary between redundant parameters and non-redundant parameters in the plurality of model parameters comprises: obtaining a total number of the model parameters; obtaining a first parameter percentage and a second parameter percentage, the first parameter percentage being smaller than the second parameter percentage; determining a first parameter number based on the first parameter percentage and the total number; determining a second parameter number based on the second parameter percentage and the total number; sorting the plurality of model parameters in ascending order to obtain a model parameter sequence; determining a first threshold value in the model parameter sequence based on the first parameter number; determining a second threshold value in the model parameter sequence based on the second parameter number; determining the first threshold value and the second threshold value as the parameter threshold value; determining a training state of a plurality of convolution kernels in the pre-trained model based on the parameter threshold value and convolution kernel parameters of the plurality of convolution kernels in the pre-trained model; determining a training state of a plurality of convolution kernels in the pre-trained model based on the parameter threshold value and convolution kernel parameters of the plurality of convolution kernels in the pre-trained model comprises: obtaining a plurality of convolution kernels in the pre-trained model and a convolution kernel parameter of each convolution kernel; adding a plurality of convolution kernel parameters to a corresponding convolution kernel parameter set based on the parameter threshold value; determining a training state of the plurality of convolution kernels in the pre-trained model based on each convolution kernel parameter set; determining a training state of the plurality of convolution kernels in the pre-trained model based on each convolution kernel parameter set comprises: for each convolution kernel, determining a parameter distribution of the convolution kernel parameters in each convolution kernel parameter set; adding the convolution kernel to a corresponding convolution kernel set based on the parameter distribution, different convolution kernel sets corresponding to different training states; setting a corresponding first hyperparameter for the convolution kernel in different training states; replacing a corresponding second hyperparameter in the pre-trained model with the first hyperparameter to obtain a fine-tuned target detection model for environmental perception of autonomous driving.
2. The model fine-tuning method of claim 1, wherein, adding a plurality of convolution kernel parameters to a corresponding convolution kernel parameter set based on the parameter threshold value comprises: determining a first threshold value and a second threshold value according to the parameter threshold value, the first threshold value being smaller than the second threshold value; for each convolution kernel parameter, if the convolution kernel parameter is less than or equal to the first threshold value, adding the convolution kernel parameter to a first convolution kernel parameter set; if the convolution kernel parameter is greater than the first threshold value and less than the second threshold value, adding the convolution kernel parameter to a second convolution kernel parameter set; if the convolution kernel parameter is greater than the second threshold value, adding the convolution kernel parameter to a third convolution kernel parameter set.
3. The model fine-tuning method of claim 1, wherein, for each convolution kernel, determining a parameter distribution of the convolution kernel parameters in each convolution kernel parameter set comprises: For each convolution kernel, a proportion of a number of convolution kernel parameters to be added to the first convolution kernel parameter set to a number of all convolution kernel parameters in the convolution kernel is determined as a first parameter distribution ratio; a proportion of a number of convolution kernel parameters to be added to the second convolution kernel parameter set to a number of all convolution kernel parameters in the convolution kernel is determined as a second parameter distribution ratio; a proportion of a number of convolution kernel parameters to be added to the third convolution kernel parameter set to a number of all convolution kernel parameters in the convolution kernel is determined as a third parameter distribution ratio; the first parameter distribution ratio, the second parameter distribution ratio and the third parameter distribution ratio are determined as the parameter distribution condition.
4. The model fine-tuning method of claim 3, wherein, adding the convolution kernel to a corresponding convolution kernel set based on the parameter distribution condition, different convolution kernel sets corresponding to different training states, including: if the first parameter distribution ratio is greater than a third threshold value, adding the convolution kernel to a first convolution kernel set, the first convolution kernel set corresponding to a first training state; if the second parameter distribution ratio is greater than a third threshold value, adding the convolution kernel to a second convolution kernel set, the second convolution kernel set corresponding to a second training state; if the third parameter distribution ratio is greater than a third threshold value, adding the convolution kernel to a third convolution kernel set, the third convolution kernel set corresponding to a third training state.
5. The model fine-tuning method of claim 1, wherein, setting corresponding first hyperparameters for the convolution kernels of different training states, including: obtaining a preset learning rate parameter and a preset regularization parameter set when pre-training a model; obtaining a first weight corresponding to the convolution kernel of each training state and a second weight corresponding to the convolution kernel of each training state; determining a learning rate corresponding to the convolution kernel of each training state based on the first weight and the preset learning rate parameter, and determining a learning rate of an optimizer based on the learning rate corresponding to the convolution kernel of each training state; determining a regularization parameter corresponding to the convolution kernel of each training state based on the second weight and the preset regularization parameter, and determining a regularization parameter of the optimizer based on the regularization parameter corresponding to the convolution kernel of each training state.
6. The model fine-tuning method of claim 1, wherein, The method further comprises: obtaining a test data set, the test data set comprising a plurality of frames of laser radar data; frame by frame, inputting the laser radar data into the pre-trained model to make the pre-trained model output activation values of a plurality of feature maps corresponding to the current frame of laser radar data; if a probability of the activation values of the plurality of feature maps corresponding to the current frame of laser radar data being equal to a preset value is greater than a fourth threshold value, switching an activation function of the plurality of feature maps corresponding to the current frame of laser radar data from a first activation function to a second activation function.
7. A model fine-tuning apparatus characterized by comprising: including: a first obtaining module for obtaining a plurality of model parameters of a pre-trained model, and determining a parameter threshold value as a boundary between redundant parameters and non-redundant parameters in a plurality of the model parameters; The parameter threshold value between the redundant parameters and the non-redundant parameters is determined in the plurality of model parameters, including: obtaining the total number of the model parameters; obtaining a preset first parameter percentage and a second parameter percentage, the first parameter percentage being less than the second parameter percentage; determining a first parameter number based on the first parameter percentage and the total number; determining a second parameter number based on the second parameter percentage and the total number; sorting the plurality of model parameters in ascending order to obtain a model parameter sequence; determining a first threshold value in the model parameter sequence based on the first parameter number; determining a second threshold value in the model parameter sequence based on the second parameter number; and determining the first threshold value and the second threshold value as the parameter threshold value; The first determining module is configured to determine the training states of the plurality of convolution kernels based on the parameter threshold value and the kernel parameters of the plurality of convolution kernels in the pre-trained model; the determination of the training states of the plurality of convolution kernels based on the parameter threshold value and the kernel parameters of the plurality of convolution kernels in the pre-trained model includes: obtaining the plurality of convolution kernels in the pre-trained model and the kernel parameters of each convolution kernel; adding the plurality of kernel parameters to corresponding kernel parameter sets based on the parameter threshold value; determining the training states of the plurality of convolution kernels in the pre-trained model based on each kernel parameter set; the determination of the training states of the plurality of convolution kernels in the pre-trained model based on each kernel parameter set includes: for each convolution kernel, determining the parameter distribution of the kernel parameters in each kernel parameter set; and adding the convolution kernel to a corresponding kernel set based on the parameter distribution, different kernel sets corresponding to different training states; The setting module is configured to set corresponding first hyperparameters for the convolution kernels in different training states; The replacement module is configured to replace corresponding second hyperparameters in the pre-trained model with the first hyperparameters to obtain a fine-tuned target detection model for environmental perception of autonomous driving.
8. An electronic device, comprising: The system includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is configured to store a computer program; The processor is configured to execute the program stored in the memory to implement the model fine-tuning method of any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program of a model fine-tuning method, and the program of the model fine-tuning method is executed by the processor to implement the steps of the model fine-tuning method of any one of claims 1-6.
Citation Information
Patent Citations
Network fault diagnosis method based on deep learning in heterogeneous network environment
CN108540330A
Model parameter adjustment method and computing device
CN113536690A