Large model rapid tuning method and device based on multi-round adversarial optimization

Abstract

The present invention relates to artificial intelligence technology, and discloses a large model fast tuning method and device based on multi-round adversarial optimization, including: dividing the acquired tuning image data set into a training set verification set and a validation set, using a pre-trained large model to predict the results of the training set, adding noise to the training set according to the gradient value of the prediction result and the standard result, obtaining an initial adversarial data set, using the large model to predict the results of the initial adversarial data, adjusting the hyperparameters of the large model according to the loss value of the adversarial prediction result and the standard result and adding noise to the training set until the number of iterations is reached, using the validation set to verify the performance of the large model, confirming whether the tuning is completed according to the verification result, and confirming that the tuned large model is obtained if the tuning is completed, and using the tuned large model to perform object recognition on the acquired monitoring image to obtain a recognition result. The present invention also proposes a large model fast tuning device based on multi-round adversarial optimization. The present invention can improve the efficiency of model tuning.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model rapid tuning method and device based on multi-round adversarial optimization. Background Art

[0002] With the rapid development of artificial intelligence technology, large models (such as language models, image recognition models, etc.) are increasingly used in various fields. However, the training and optimization process of large models is complex and time-consuming. Especially when faced with complex and changing task requirements, how to quickly and effectively tune large models has become an urgent problem to be solved.

[0003] Currently, large model tuning methods mainly rely on traditional iterative training and hyperparameter adjustment strategies. Although these methods can improve model performance to a certain extent, they are inefficient and costly. Summary of the invention

[0004] The present invention provides a large model rapid tuning method and device based on multi-round adversarial optimization, the main purpose of which is to solve the problem of high efficiency and low cost of existing large model tuning methods.

[0005] To achieve the above object, the present invention provides a large model fast tuning method based on multi-round adversarial optimization, comprising:

[0006] Dividing the pre-acquired tuning image data set into a training set and a validation set, and using the pre-trained large model to predict the results of the training set to obtain a set of prediction results;

[0007] Obtaining a gradient value between each prediction result in the prediction result set and a preset standard result to obtain a gradient value set, and adding noise to the training set according to the gradient value set to obtain an initial adversarial data set;

[0008] Using the large model to predict the result of each adversarial sample in the initial adversarial data set, obtaining an adversarial prediction result set, and calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result, to obtain a loss value set;

[0009] Adjusting the hyperparameters of the large model according to the loss value set, and adding noise to the training set according to the loss value set to obtain an iterative adversarial data set;

[0010] After using the large model to predict the result of each adversarial sample in the iterative adversarial data set to obtain an adversarial prediction result set, returning to the step of calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result to obtain a loss value set;

[0011] When the large model reaches a preset number of iterations, the performance of the large model is verified using the verification set;

[0012] If the tuned large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuned image data set into a training set and a verification set is returned to, and the result of the training set is predicted using the pre-trained large model to obtain a set of prediction results;

[0013] If the tuned large model passes the performance verification, it is confirmed that the tuning of the large model is completed, and the tuned large model is obtained;

[0014] The tuned large model is used to perform object recognition on the pre-acquired monitoring image to obtain an object recognition result.

[0015] Optionally, the using of the pre-trained large model to predict the results of the training set to obtain a set of prediction results includes:

[0016] Performing feature extraction on the training set to obtain training set features;

[0017] Using the convolution layer of the large model to extract local features of each data in the training set, to obtain a convolution feature set;

[0018] Using the pooling layer of the large model to extract the local average value of each convolution feature in the convolution feature set, to obtain a pooling feature set;

[0019] Using the activation layer of the large model to add a nonlinear factor to each pooling feature in the pooling feature set, to obtain an activated feature set;

[0020] The fully connected layer of the large model is used to output the prediction result of each activated feature in the activated feature set according to the preset weight parameters and bias items to obtain a prediction result set.

[0021] Optionally, obtaining a gradient value between each prediction result in the prediction result set and a preset standard result to obtain a gradient value set includes:

[0022] The gradient value is calculated using the following formula:

[0023]

[0024] Among them, L i is the i-th gradient value in the obtained gradient value set, N is the number of samples in the prediction result set, y i is the i-th sample in the prediction result set, is the standard result corresponding to the i-th sample in the prediction result set.

[0025] Optionally, adding noise to the training set according to the gradient value set to obtain an initial adversarial data set includes:

[0026] Noise is added to the training set using the following formula:

[0027]

[0028] Among them, P i is the i-th data in the initial adversarial training set, Z i is the i-th data in the training set, L i is the i-th gradient value in the obtained gradient value set, α is a scaling factor for controlling the noise size, σ is the standard deviation of the gradient value set, and μ is the average value of the gradient value set.

[0029] Optionally, the calculating a loss value between each adversarial prediction result in the adversarial prediction result set and a preset standard result includes:

[0030] The loss value is calculated using the following formula:

[0031]

[0032] Among them, D i is the i-th loss value in the loss value set, x i is the ith adversarial prediction result in the adversarial prediction result set, is the standard result corresponding to the ith adversarial prediction result in the adversarial prediction result set.

[0033] Optionally, adjusting the hyper parameters of the large model according to the loss value set includes:

[0034] Calculate the average value of the loss value set to obtain an average loss value;

[0035] Using the average loss value to adjust the learning rate included in the hyperparameter to obtain an updated learning rate;

[0036] The weight parameters included in the hyperparameters are updated according to the average loss value and the updated learning rate using the following formula to obtain updated weight parameters:

[0037]

[0038] in, is the updated weight parameter, θ is the weight parameter, η is the updated learning rate, is the average loss value.

[0039] Optionally, adding noise to the training set according to the loss value set to obtain an iterative adversarial data set includes:

[0040] Noise is added to the training set using the following formula:

[0041]

[0042] Among them, Q i is the i-th data in the iterative adversarial dataset, Z i is the i-th data in the training set, β is the adjustment factor for controlling the noise size, D i is the i-th loss value in the loss value set, is the standard deviation of the loss value set.

[0043] Optionally, the using the verification set to perform performance verification on the large model includes:

[0044] Using the large model to predict the results of each data in the validation set, to obtain a set of prediction results of the validation set;

[0045] Calculate the loss value according to the validation set prediction result set and the preset standard result of the validation to obtain the validation set loss value;

[0046] If the validation set loss value is greater than or equal to a preset loss value threshold, it is determined that the large model has failed the performance verification;

[0047] If the validation set loss value is less than the loss value threshold, it is determined that the large model has passed the performance verification.

[0048] Optionally, the calculating the loss value according to the validation set prediction result set and the preset standard result of the validation to obtain the validation set loss value includes:

[0049] The validation set loss value is calculated using the following formula:

[0050]

[0051] Among them, X is the loss value of the validation set, K is the number of samples in the validation set, and R i is the i-th prediction result in the prediction result set of the validation set, is the true result corresponding to the ith prediction result in the validation set prediction result set.

[0052] In order to solve the above problems, the present invention also provides a large model fast tuning device based on multi-round adversarial optimization, the device comprising:

[0053] A model prediction module is used to divide the pre-acquired tuning image data set into a training set and a validation set, and use the pre-trained large model to predict the results of the training set to obtain a set of prediction results;

[0054] A noise adding module, used for obtaining a gradient value between each prediction result in the prediction result set and a preset standard result, obtaining a gradient value set, and adding noise to the training set according to the gradient value set to obtain an initial adversarial data set;

[0055] A multi-round optimization module, used for using the large model to predict the results of each adversarial sample in the initial adversarial data set to obtain an adversarial prediction result set, calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result to obtain a loss value set, adjusting the hyperparameters of the large model according to the loss value set, and adding noise to the training set according to the loss value set to obtain an iterative adversarial data set, using the large model to predict the results of each adversarial sample in the iterative adversarial data set to obtain the adversarial prediction result set, and returning to the step of calculating the loss value of each adversarial prediction result in the adversarial prediction result set and the preset standard result to obtain the loss value set;

[0056] The performance judgment module performs performance verification on the large model using the verification set when the large model reaches a preset number of iterations. If the tuned large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuned image data set into a training set and a verification set is returned to, and the result of the training set is predicted using the pre-trained large model to obtain a set of prediction results. If the tuned large model passes the performance verification, it is confirmed that the tuning of the large model is completed, and the tuned large model is obtained;

[0057] The image recognition module uses the tuned large model to perform object recognition on the pre-acquired monitoring image to obtain an object recognition result.

[0058] The embodiment of the present invention divides a pre-acquired tuning image dataset into a training set and a validation set, uses a pre-trained large model to predict the results of the training set to obtain a prediction result set, obtains the gradient value of each prediction result in the prediction result set and a preset standard result to obtain a gradient value set, adds noise to the training set according to the gradient value set to obtain an initial adversarial dataset, uses the large model to predict the results of each adversarial sample in the initial adversarial dataset to obtain an adversarial prediction result set, calculates the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result to obtain a loss value set, adjusts the hyperparameters of the large model according to the loss value set, and adds noise to the training set according to the loss value set to obtain an iterative adversarial dataset, and uses the large model to perform noise prediction on the iterative adversarial dataset. After predicting the result of each adversarial sample in the adversarial data set and obtaining the adversarial prediction result set, the step of calculating the loss value of each adversarial prediction result in the adversarial prediction result set and the preset standard result and obtaining the loss value set is returned. When the large model reaches the preset number of iterations, the performance of the large model is verified using the verification set. If the tuned large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuned image data set into a training set and a verification set is returned. The training set is predicted using the pre-trained large model to obtain the prediction result set. If the tuned large model passes the performance verification, it is confirmed that the tuning of the large model is completed and the tuned large model is obtained. The tuned large model is used to perform object recognition on the pre-acquired surveillance image to obtain the object recognition result. Therefore, the large model fast tuning method, device, electronic device and computer-readable storage medium based on multi-round adversarial optimization proposed in the present invention can solve the problem of high efficiency and low cost of the existing large model tuning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a process flow of a large model rapid optimization method based on multi-round adversarial optimization provided by an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a process for predicting results of a training set using a large model provided in one embodiment of the present invention;

[0061] Figure 3 A functional module diagram of a large model rapid tuning device based on multi-round adversarial optimization provided by one embodiment of the present invention.

[0062] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0064] The embodiment of the present application provides a large model rapid tuning method based on multi-round adversarial optimization. The execution subject of the large model rapid tuning method based on multi-round adversarial optimization includes but is not limited to at least one of the electronic devices that can be configured to execute the method provided in the embodiment of the present application, such as a server, a terminal, etc. In other words, the large model rapid tuning method based on multi-round adversarial optimization can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0065] Reference Figure 1 FIG. 1 is a flow chart of a large model fast tuning method based on multi-round adversarial optimization provided by an embodiment of the present invention. In this embodiment, the large model fast tuning method based on multi-round adversarial optimization includes:

[0066] S1. Divide the pre-acquired tuning image data set into a training set and a validation set, and use the pre-trained large model to predict the results of the training set to obtain a set of prediction results.

[0067] In the embodiment of the present invention, the optimized image data set is a data set subsequently used to optimize the large model.

[0068] Furthermore, each data in the tuned image data set has an accurate label, and the image data has an accurate label of the image content.

[0069] In the embodiment of the present invention, the pre-acquired tuning image data set is divided into a training set and a verification set according to a preset ratio, and the preset ratio may be 70% for the training set and 30% for the verification set.

[0070] In an embodiment of the present invention, the large model is a neural network model, including but not limited to a convolutional neural network model and a recurrent neural network model.

[0071] In the embodiment of the present invention, referring to Figure 2As shown, a schematic diagram of a process of using a large model to predict results on a training set is provided in an embodiment of the present invention. The method of using a pre-trained large model to predict results on the training set to obtain a set of prediction results includes:

[0072] S21, extracting features from the training set to obtain training set features;

[0073] S22, using the convolution layer of the large model to extract local features of each data in the training set to obtain a convolution feature set;

[0074] S23, using the pooling layer of the large model to extract the local average value of each convolution feature in the convolution feature set to obtain a pooling feature set;

[0075] S24, using the activation layer of the large model to add a nonlinear factor to each pooling feature in the pooling feature set to obtain an activated feature set;

[0076] S25. Utilize the fully connected layer of the large model to output the prediction result of each activated feature in the activated feature set according to preset weight parameters and bias items to obtain a prediction result set.

[0077] In detail, the feature extraction of the training set is a step of identifying, selecting and converting information from the original data that is helpful for the neural network model to understand, which can reduce the dimension of the data and highlight the important information of the data.

[0078] In detail, the feature extraction for the training set may be performed by extracting features from the data in the training set through a gradient histogram.

[0079] In detail, using the convolution layer of the large model to extract the local features of each data in the training set refers to using a preset convolution kernel to move on the data, calculating the dot product of each moved convolution kernel and the covered part of the data, summing all the calculated dot product results, and obtaining the convolution feature.

[0080] In detail, using the activation layer of the large model to add a nonlinear factor to each pooling feature in the pooling feature set refers to using a pre-set rectangular window to move on the convolution feature, calculating the average value of the data of the covered part of the rectangular window after each movement, replacing the data of the original covered area with the average value, and obtaining the pooling feature.

[0081] In detail, using the activation layer of the large model to add a nonlinear factor to each pooled feature in the pooled feature set refers to using a pre-set activation function to add a nonlinear factor to the pooled feature to improve the model's learning ability for complex features. Common activation functions include Sigmoid function, Softmax function and ReLU function.

[0082] In detail, the weight parameters are variables that connect each pair of neurons in a neural network and are used to transfer and process information between neurons. Each connection has a weight parameter, and these values ​​are constantly updated during the training process.

[0083] In detail, the bias term is a constant added to the input of the neural network neuron activation function, which adds a constant offset to each neuron so that the model can better fit the data. Without the bias term, the neural network can only adjust the decision boundary near the origin, which limits the expressive power of the model.

[0084] In an embodiment of the present invention, a pre-acquired tuning image data set is divided into a training set and a validation set, and a pre-trained large model is used to predict the results of the training set to obtain a prediction result set, thereby improving the efficiency of subsequent calculation of the gradient value set.

[0085] S2. Obtain the gradient value of each prediction result in the prediction result set and the preset standard result to obtain a gradient value set, and add noise to the training set according to the gradient value set to obtain an initial adversarial data set.

[0086] In the embodiment of the present invention, the gradient value is a vector, indicating the direction in which the rate of change of the function at a certain point along all directions is the fastest.

[0087] In the embodiment of the present invention, the preset standard result refers to a standard labeling performed on each data in the tuning image data set.

[0088] In the embodiment of the present invention, adding noise to the training set according to the gradient value set refers to adding noise to the corresponding data in the training set according to the gradient value of the prediction result of the large model for each data in the training set.

[0089] In an embodiment of the present invention, the gradient value of the prediction result of each data in the training set according to the large model is used to add noise to the corresponding data in the training set. This is because the gradient value of the prediction result reflects the weakness of the large model to a certain extent. Therefore, adding noise to the training set according to the gradient value can improve the effect of adversarial training.

[0090] In the embodiment of the present invention, the step of obtaining a gradient value between each prediction result in the prediction result set and a preset standard result to obtain a gradient value set includes:

[0091] The gradient value is calculated using the following formula:

[0092]

[0093] Among them, L i is the i-th gradient value in the obtained gradient value set, N is the number of samples in the prediction result set, y i is the i-th sample in the prediction result set, is the standard result corresponding to the i-th sample in the prediction result set.

[0094] In the embodiment of the present invention, adding noise to the training set according to the gradient value set to obtain an initial adversarial data set includes:

[0095] Noise is added to the training set using the following formula:

[0096]

[0097] Among them, P i is the i-th data in the initial adversarial training set, Z i is the i-th data in the training set, L i is the i-th gradient value in the obtained gradient value set, α is a scaling factor for controlling the noise size, σ is the standard deviation of the gradient value set, and μ is the average value of the gradient value set.

[0098] In an embodiment of the present invention, a gradient value set is obtained by obtaining the gradient value of each prediction result in the prediction result set and a preset standard result, thereby improving the efficiency of adding noise to the training set. By adding noise to the training set according to the gradient value set, an initial adversarial data set is obtained, thereby improving the effect of subsequent multiple rounds of adversarial optimization of the large model.

[0099] S3. Use the large model to predict the results of each adversarial sample in the initial adversarial data set to obtain an adversarial prediction result set, calculate the loss value of each adversarial prediction result in the adversarial prediction result set and the preset standard result to obtain a loss value set.

[0100] In the embodiment of the present invention, the step of using the large model to predict the results of each adversarial sample in the initial adversarial data set is the same as the step of using the pre-trained large model to predict the results of the training set to obtain a set of prediction results, which will not be repeated here.

[0101] In the embodiment of the present invention, the calculation of the loss value between each adversarial prediction result in the adversarial prediction result set and a preset standard result is performed using a preset loss function.

[0102] In detail, the loss value is an indicator used to evaluate the performance of the model when training a neural network. Specifically, it reflects the degree of fit of the model to the training data under the current parameter settings.

[0103] Specifically, the calculation of the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result includes:

[0104] The loss value is calculated using the following formula:

[0105]

[0106] Among them, D i is the i-th loss value in the loss value set, x i is the ith adversarial prediction result in the adversarial prediction result set, is the standard result corresponding to the ith adversarial prediction result in the adversarial prediction result set.

[0107] In an embodiment of the present invention, the large model is used to predict the results of each adversarial sample in the initial adversarial data set to obtain an adversarial prediction result set, thereby improving the efficiency of subsequent calculation of loss values. By calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result, a loss value set is obtained, thereby improving the efficiency of subsequent iterative adversarial data sets.

[0108] S4. Adjust the hyperparameters of the large model according to the loss value set, and add noise to the training set according to the loss value set to obtain an iterative adversarial data set.

[0109] In the embodiment of the present invention, the hyperparameters include weight parameters and learning rates.

[0110] In detail, the learning rate is an important parameter in the neural network model. The learning rate determines the amplitude of the weight parameter update in the gradient descent direction of the loss function. If the learning rate is too large, the model may oscillate or even diverge near the optimal solution; if the learning rate is too small, the training speed of the model will be very slow, or even fall into a local minimum. Usually a small initial learning rate is set, and then adjusted as needed during the training process.

[0111] In the embodiment of the present invention, adjusting the hyperparameters of the large model according to the loss value set includes:

[0112] Calculate the average value of the loss value set to obtain an average loss value;

[0113] Using the average loss value to adjust the learning rate included in the hyperparameter to obtain an updated learning rate;

[0114] The weight parameters included in the hyperparameters are updated according to the average loss value and the updated learning rate using the following formula to obtain updated weight parameters:

[0115]

[0116] in, is the updated weight parameter, θ is the weight parameter, η is the updated learning rate, is the average loss value.

[0117] In the embodiment of the present invention, since the loss value reflects the degree of fit of the model to the training data under the current parameter settings, noise can be added to the data according to the loss value to improve the effect of adversarial training.

[0118] In the embodiment of the present invention, adding noise to the training set according to the loss value set to obtain an iterative adversarial data set includes:

[0119] Noise is added to the training set using the following formula:

[0120]

[0121] Among them, Q i is the i-th data in the iterative adversarial dataset, Z i is the i-th data in the training set, β is the adjustment factor for controlling the noise size, D i is the i-th loss value in the loss value set, is the standard deviation of the loss value set.

[0122] In an embodiment of the present invention, the performance of the model is improved by adjusting the hyperparameters of the large model according to the loss value set, and by adding noise to the training set according to the loss value set, an iterative adversarial data set is obtained, thereby improving the effect of subsequent optimization of the model.

[0123] S5. Use the large model to predict the results of each adversarial sample in the iterative adversarial data set. After obtaining a set of adversarial prediction results, return to the step of calculating the loss value of each adversarial prediction result in the set of adversarial prediction results and a preset standard result to obtain a set of loss values.

[0124] In an embodiment of the present invention, after obtaining the iterative adversarial data set, the large model is used again to predict the results of each iterative adversarial data in the iterative adversarial data set, and then the step of calculating the loss value between the predicted result and the actual result is returned to generate an iterative adversarial data set by adjusting the hyperparameters according to the loss value and adding noise to the training set until the preset number of iterations is reached, thereby achieving multi-round adversarial optimization of the large model.

[0125] In detail, except for the first round, in each round of adversarial optimization, the iterative adversarial data set is obtained through the loss value set obtained in the previous round, that is, the iterative adversarial data set of each round is different, which can improve the optimization effect of large models.

[0126] In an embodiment of the present invention, by using the large model to predict the results of each adversarial sample in the iterative adversarial data set, after obtaining a set of adversarial prediction results, the step of calculating the loss value of each adversarial prediction result in the set of adversarial prediction results and a preset standard result to obtain a set of loss values ​​is returned, thereby realizing multiple rounds of adversarial optimization of the large model and improving the tuning effect of the large model.

[0127] S6. When the large model reaches a preset number of iterations, the performance of the large model is verified using the verification set.

[0128] In the embodiment of the present invention, the preset number of iterations needs to be set according to requirements. Too low an iteration number will lead to poor model optimization effect, and too high an iteration number may cause overfitting of the model.

[0129] In the embodiment of the present invention, the performance verification of the large model using the verification set refers to using the large model to predict the results of the data in the verification set, and then judging whether the performance of the model meets the standard based on the difference between the predicted results and the standard results.

[0130] In the embodiment of the present invention, the use of the verification set to perform performance verification on the large model includes:

[0131] Using the large model to predict the results of each data in the validation set, to obtain a set of prediction results of the validation set;

[0132] Calculate the loss value according to the validation set prediction result set and the preset standard result of the validation to obtain the validation set loss value;

[0133] If the validation set loss value is greater than or equal to a preset loss value threshold, it is determined that the large model has failed the performance verification;

[0134] If the validation set loss value is less than the loss value threshold, it is determined that the large model has passed the performance verification.

[0135] In detail, the loss value is calculated according to the validation set prediction result set and the preset standard result of the validation to obtain the validation set loss value, including:

[0136] The validation set loss value is calculated using the following formula:

[0137]

[0138] Among them, X is the loss value of the validation set, K is the number of samples in the validation set, and R i is the i-th prediction result in the prediction result set of the validation set, is the true result corresponding to the ith prediction result in the validation set prediction result set.

[0139] In the embodiment of the present invention, when the large model reaches a preset number of iterations, the performance of the large model is verified by using the verification set to ensure the accuracy of the large model.

[0140] S7. Determine whether the large model passes the performance verification.

[0141] If the tuned large model fails the performance verification, execute S8, obtain a new adjustment data set, return to the step of dividing the pre-acquired tuned image data set into a training set and a verification set, and use the pre-trained large model to predict the results of the training set to obtain a set of prediction results.

[0142] In the embodiment of the present invention, when the tuned large model fails to pass the performance verification, it indicates that the tuning effect of the large model does not meet expectations, and therefore it is necessary to re-perform multiple rounds of adversarial optimization on the large model.

[0143] In detail, when re-performing multiple rounds of adversarial optimization on the large model, it is necessary to obtain a new tuning image dataset to improve the effect of the multiple rounds of adversarial optimization.

[0144] In an embodiment of the present invention, when the tuned large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuned image data set into a training set and a verification set is returned, and the result of the training set is predicted using the pre-trained large model to obtain a set of prediction results, thereby improving the effect of tuning the large model.

[0145] If the tuned large model passes the performance verification, S9 is executed to confirm that the tuning of the large model is completed, and the tuned large model is obtained.

[0146] In the embodiment of the present invention, when the tuned large model passes the performance verification, it means that the effect of multiple rounds of adversarial optimization on the large model has reached the expectation, and it is confirmed that the model optimization is completed.

[0147] In the embodiment of the present invention, when the tuned large model passes the performance verification, it is confirmed that the tuning of the large model is completed, and the tuned large model is obtained, which improves the efficiency and effect of performing multiple rounds of adversarial optimization on the large model.

[0148] S10: Use the tuned large model to perform object recognition on the pre-acquired monitoring image to obtain an object recognition result.

[0149] In the embodiment of the present invention, the pre-acquired monitoring image refers to the monitoring image obtained by capturing the monitoring video of the preset monitoring scene at a preset frequency. The preset frequency needs to be set according to the demand. A too high frequency can increase the amount of information contained in the monitoring image and improve the subsequent recognition accuracy, but increase the amount of calculation. A too low frequency can improve the processing efficiency, but will reduce the accuracy of subsequent recognition.

[0150] In the embodiment of the present invention, the object recognition is performed on the pre-acquired surveillance image using the tuned large model to obtain the object recognition result in order to judge whether there is human intrusion and item loss in the surveillance scene according to the object recognition result, thereby improving the security of the surveillance scene.

[0151] like Figure 3 As shown, it is a functional module diagram of a large model rapid tuning device based on multi-round adversarial optimization provided by an embodiment of the present invention.

[0152] The large model fast tuning device 100 based on multi-round adversarial optimization of the present invention can be installed in an electronic device. According to the functions implemented, the large model fast tuning device 100 based on multi-round adversarial optimization can include a model prediction module 101, a noise addition module 102, a multi-round optimization module 103, a performance judgment module 104 and an image recognition module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0153] In this embodiment, the functions of each module / unit are as follows:

[0154] The model prediction module 101 is used to divide the pre-acquired tuning image data set into a training set and a validation set, and use the pre-trained large model to perform result prediction on the training set to obtain a prediction result set;

[0155] The noise adding module 102 is used to obtain the gradient value between each prediction result in the prediction result set and the preset standard result to obtain a gradient value set, and add noise to the training set according to the gradient value set to obtain an initial adversarial data set;

[0156] The multi-round optimization module 103 is used to use the large model to predict the results of each adversarial sample in the initial adversarial data set to obtain an adversarial prediction result set, calculate the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result to obtain a loss value set, adjust the hyperparameters of the large model according to the loss value set, and add noise to the training set according to the loss value set to obtain an iterative adversarial data set, use the large model to predict the results of each adversarial sample in the iterative adversarial data set, and after obtaining the adversarial prediction result set, return to the step of calculating the loss value of each adversarial prediction result in the adversarial prediction result set and the preset standard result to obtain the loss value set;

[0157] The performance judgment module 104 performs performance verification on the large model using the verification set when the large model reaches a preset number of iterations. If the tuned large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuned image data set into a training set and a verification set is returned to, and the result of the training set is predicted using the pre-trained large model to obtain a set of prediction results. If the tuned large model passes the performance verification, it is confirmed that the tuning of the large model is completed, and the tuned large model is obtained;

[0158] The image recognition module 105 uses the tuned large model to perform object recognition on the pre-acquired monitoring image to obtain an object recognition result.

[0159] In detail, each module described in the large model fast tuning device 100 based on multi-round adversarial optimization in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The same technical means are used as the large model rapid tuning method based on multi-round adversarial optimization described in , and can produce the same technical effects, so I will not go into details here.

[0160] In the embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0161] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0163] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0164] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0165] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0166] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A large model fast tuning method based on multi-round adversarial optimization, characterized in that: The method comprises: Dividing the pre-acquired tuning image data set into a training set and a validation set, and using the pre-trained large model to predict the results of the training set to obtain a set of prediction results; Obtaining a gradient value between each prediction result in the prediction result set and a preset standard result to obtain a gradient value set, adding noise to the training set according to the gradient value set to obtain an initial adversarial data set, wherein adding noise to the training set according to the gradient value set includes: Noise is added to the training set using the following formula: Among them, P i is the i-th data in the initial adversarial data set, Z i is the i-th data in the training set, L i is the i-th gradient value in the obtained gradient value set, α is a scaling factor for controlling the noise size, σ is the standard deviation of the gradient value set, and μ is the average value of the gradient value set; Using the large model to predict the result of each adversarial sample in the initial adversarial data set, obtaining an adversarial prediction result set, and calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result, to obtain a loss value set; Adjusting the hyperparameters of the large model according to the loss value set, and adding noise to the training set according to the loss value set to obtain an iterative adversarial data set; After using the large model to predict the result of each adversarial sample in the iterative adversarial data set to obtain an adversarial prediction result set, returning to the step of calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result to obtain a loss value set; When the large model reaches a preset number of iterations, the performance of the large model is verified using the verification set; If the large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuning image data set into a training set and a verification set is returned to, and the result of the training set is predicted using the pre-trained large model to obtain a set of prediction results; If the large model passes the performance verification, it is confirmed that the large model tuning is completed and the large model is obtained; The large model is used to perform object recognition on the pre-acquired monitoring image to obtain an object recognition result.

2. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 1 is characterized in that: The method of using the pre-trained large model to predict the results of the training set to obtain a set of prediction results includes: Performing feature extraction on the training set to obtain training set features; Using the convolution layer of the large model to extract local features of each data in the training set, obtaining a convolution feature set; Using the pooling layer of the large model to extract the local average value of each convolution feature in the convolution feature set, to obtain a pooling feature set; Using the activation layer of the large model to add a nonlinear factor to each pooling feature in the pooling feature set, to obtain an activated feature set; The fully connected layer of the large model is used to output the prediction result of each activated feature in the activated feature set according to the preset weight parameters and bias items to obtain a prediction result set.

3. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 2 is characterized in that: The step of obtaining a gradient value between each prediction result in the prediction result set and a preset standard result to obtain a gradient value set includes: The gradient value is calculated using the following formula: Among them, L i is the i-th gradient value in the obtained gradient value set, N is the number of samples in the prediction result set, y i is the i-th sample in the prediction result set, is the standard result corresponding to the i-th sample in the prediction result set.

4. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 1, characterized in that: The calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result includes: The loss value is calculated using the following formula: Among them, D i is the i-th loss value in the loss value set, x i is the ith adversarial prediction result in the adversarial prediction result set, is the standard result corresponding to the ith adversarial prediction result in the adversarial prediction result set.

5. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 4 is characterized in that: The step of adjusting the hyper parameters of the large model according to the loss value set includes: Calculate the average value of the loss value set to obtain an average loss value; Using the average loss value to adjust the learning rate included in the hyperparameter to obtain an updated learning rate; The weight parameters included in the hyperparameters are updated according to the average loss value and the updated learning rate using the following formula to obtain updated weight parameters: in, is the updated weight parameter, θ is the weight parameter, η is the updated learning rate, is the average loss value.

6. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 5, characterized in that: The adding noise to the training set according to the loss value set to obtain an iterative adversarial data set includes: Noise is added to the training set using the following formula: Among them, Q i is the i-th data in the iterative adversarial dataset, Z i is the i-th data in the training set, β is the adjustment factor for controlling the noise size, D i is the i-th loss value in the loss value set, is the standard deviation of the loss value set.

7. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 6, characterized in that: The using the verification set to perform performance verification on the large model includes: Using the large model to predict the results of each data in the validation set, to obtain a set of prediction results of the validation set; Calculate the loss value according to the validation set prediction result set and the preset standard result of the validation to obtain the validation set loss value; If the validation set loss value is greater than or equal to a preset loss value threshold, it is determined that the large model has failed the performance verification; If the validation set loss value is less than the loss value threshold, it is determined that the large model has passed the performance verification.

8. The large model fast tuning method based on multi-round adversarial optimization as claimed in claim 7, characterized in that: The step of calculating the loss value according to the validation set prediction result set and the preset standard result of the validation to obtain the validation set loss value includes: The validation set loss value is calculated using the following formula: Among them, X is the loss value of the validation set, K is the number of samples in the validation set, and R i is the i-th prediction result in the prediction result set of the validation set, is the true result corresponding to the ith prediction result in the validation set prediction result set.

9. A large model fast tuning device based on multi-round adversarial optimization, characterized in that: The device comprises: A model prediction module is used to divide the pre-acquired tuning image data set into a training set and a validation set, and use the pre-trained large model to predict the results of the training set to obtain a set of prediction results; A noise adding module is used to obtain the gradient value of each prediction result in the prediction result set and the preset standard result, obtain the gradient value set, add noise to the training set according to the gradient value set, and obtain the initial adversarial data set, wherein the adding noise to the training set according to the gradient value set includes: Noise is added to the training set using the following formula: Among them, P i is the i-th data in the initial adversarial data set, Z i is the i-th data in the training set, L i is the i-th gradient value in the obtained gradient value set, α is a scaling factor for controlling the noise size, σ is the standard deviation of the gradient value set, and μ is the average value of the gradient value set; A multi-round optimization module, used for using the large model to predict the results of each adversarial sample in the initial adversarial data set to obtain an adversarial prediction result set, calculating the loss value of each adversarial prediction result in the adversarial prediction result set and a preset standard result to obtain a loss value set, adjusting the hyperparameters of the large model according to the loss value set, and adding noise to the training set according to the loss value set to obtain an iterative adversarial data set, using the large model to predict the results of each adversarial sample in the iterative adversarial data set to obtain the adversarial prediction result set, and returning to the step of calculating the loss value of each adversarial prediction result in the adversarial prediction result set and the preset standard result to obtain the loss value set; The performance judgment module performs performance verification on the large model using the verification set when the large model reaches a preset number of iterations. If the large model fails the performance verification, a new adjustment data set is obtained, and the step of dividing the pre-acquired tuning image data set into a training set and a verification set is returned to, and the result of the training set is predicted using the pre-trained large model to obtain a set of prediction results. If the large model passes the performance verification, it is confirmed that the tuning of the large model is completed, and the large model is obtained. The image recognition module uses the large model to perform object recognition on the pre-acquired monitoring image to obtain an object recognition result.

Images