Model adjustment method and device and electronic equipment

Abstract

The embodiment of the invention provides a model adjustment method and device and electronic equipment, and relates to the technical field of machine learning, and the method comprises the steps: determining a to-be-adjusted first model parameter of a target image recognition model, and a second model parameter of a pre-trained contrast learning model; obtaining a plurality of weight parameter groups, and performing weighted summation processing on the first model parameter and the second model parameter by using each weight parameter group to obtain a plurality of target model parameters; replacing the first model parameter with each target model parameter to obtain each candidate image recognition model; inputting a test image into each candidate image recognition model to obtain an output result, and determining an evaluation value corresponding to the candidate image recognition model based on the output result and the label of the test image; and selecting the candidate image recognition model with the maximum evaluation value as a target image recognition model matched with the second business scene. According to the scheme, the application generalization of the image recognition model can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a model adjustment method, device and electronic device. Background Art

[0002] In order to maintain the network environment of the network platform, images or videos uploaded to the platform need to be reviewed to avoid spreading images or videos that may cause bad guidance through the platform, for example, spreading images of target objects such as tattoos and smoke. Usually, images are recognized using a pre-trained image recognition model for image recognition, that is, the presentation state of the image for the object is recognized, such as the existence of an object in the image recognition.

[0003] In the related art, a pre-trained contrastive learning model, such as a pre-trained CLIP (Contrastive Language-Image Pre-Training) model, is used as an image recognition model. Specifically, when it is necessary to identify the presentation state of a target object, a sample prompt word and a sample image related to the target object are usually used to perform targeted training on the pre-trained image recognition model. This targeted training enables the trained image recognition model to identify the presentation state of the target object on the input image based on the output result of the image recognition model without the need to input a prompt word during subsequent applications. The output result of the image recognition model is the similarity between the sample prompt word used during model training and the input image. The similarity of any prompt word to an image represents the similarity between the features of the prompt word and the features of the image.

[0004] However, this will result in the trained image recognition model only having a high recognition accuracy rate for images with the same business scenario as the sample image and containing the target object, but having a low recognition accuracy rate for images with different business scenarios from the sample image but containing the target object. For example, the trained image recognition model has a high recognition accuracy rate for images with tattoos in a farm scenario, but has a low recognition accuracy rate for images with tattoos in a coffee shop scenario, thus reducing the application generalization of the image recognition model.

[0005] Based on this, how to improve the recognition accuracy of the image recognition model for the same object in any business scenario in order to improve the application generalization of the image recognition model has become a technical problem that urgently needs to be solved. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a model adjustment method, device and electronic device to improve the recognition accuracy of the image recognition model for the same object in any business scenario, thereby improving the application generalization of the image recognition model. The specific technical solution is as follows:

[0007] In a first aspect provided by an embodiment of the present application, first, a model adjustment method is provided, the method comprising:

[0008] Determine a first model parameter to be adjusted of a target image recognition model and a second model parameter of a pre-trained contrastive learning model; wherein the target image recognition model is an image recognition model for identifying a target object obtained by training the contrastive learning model based on sample images in a first business scenario; the first model parameter and the second model parameter are parameters having the same function in an image recognition process;

[0009] Acquire multiple weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain a target model parameter corresponding to the weight parameter group; wherein each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter; and the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value;

[0010] For each target model parameter, using the target model parameter to replace the first model parameter of the target image recognition model, to obtain each candidate image recognition model;

[0011] For each candidate image recognition model, a test image is input into the candidate image recognition model to obtain an output result corresponding to the test image, and based on the output result corresponding to the test image and a label of the test image, an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model is determined; wherein the test image is an image under the second business scenario and is provided with a label, and the label is used to characterize the presentation state of the target object;

[0012] A candidate image recognition model with the largest evaluation value among the candidate image recognition models is selected as a target image recognition model matching the second business scenario.

[0013] In a second aspect provided by an embodiment of the present application, a model adjustment device is also provided, the device comprising:

[0014] A parameter determination module, used to determine a first model parameter to be adjusted for a target image recognition model, and a second model parameter of a pre-trained contrastive learning model; wherein the target image recognition model is an image recognition model for identifying a target object obtained by training the contrastive learning model based on sample images in a first business scenario; the first model parameter and the second model parameter are parameters having the same function in an image recognition process;

[0015] a parameter processing module, configured to obtain a plurality of weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain a target model parameter corresponding to the weight parameter group; wherein each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter; and the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value;

[0016] A parameter updating module, for replacing the first model parameter of the target image recognition model with the target model parameter for each target model parameter, to obtain each candidate image recognition model;

[0017] An evaluation value determination module is used to input a test image into each candidate image recognition model, obtain an output result corresponding to the test image, and determine an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model based on the output result corresponding to the test image and a label of the test image; wherein the test image is an image in a second business scenario and is provided with a label, and the label is used to characterize the presentation state of the target object;

[0018] The model determination module is used to select the candidate image recognition model with the largest evaluation value among the candidate image recognition models as the target image recognition model matching the second business scenario.

[0019] In the third aspect provided by the embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement any model adjustment method provided in the first aspect when executing the program stored in the memory.

[0020] In another aspect provided by the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any model adjustment method provided in the first aspect is implemented.

[0021] In another aspect provided by the embodiments of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any model adjustment method provided in the first aspect above.

[0022] The solution provided in the embodiment of the present application can, when adjusting the model, fuse the model parameters of the target image recognition model and the model parameters of the corresponding contrast learning model according to the weight parameter group, so as to provide the target image recognition model with the data features required for image recognition of the same object in a second business scenario through parameter fusion, thereby improving the recognition accuracy of the image recognition model for the same object in any business scenario, so as to improve the application generalization of the image recognition model.

[0023] Furthermore, the adjusted target image recognition model is a candidate image recognition model with the largest evaluation value determined based on the output results of the model and the label of the test image. That is to say, the adjusted target image recognition model is the recognition model with the highest recognition accuracy among all candidate recognition models when performing image recognition on images containing target objects in the second business scenario. Therefore, applying a model adjustment method provided in an embodiment of the present application can improve the accuracy of the adjusted target image recognition model when performing recognition on images containing target objects in the second business scenario while ensuring the recognition accuracy of the adjusted target image recognition model for images containing target objects in the first business scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0025] Figure 1 A schematic diagram of the classification process of the existing CLIP model;

[0026] Figure 2 A schematic diagram of a flow chart of a first model adjustment method provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of a flow chart of a second model adjustment method provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a training process of a target image recognition model provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the image recognition process of a contrastive learning model;

[0030] Figure 6This is a schematic diagram of the model structure of a traditional deep learning recognition model;

[0031] Figure 7 A schematic diagram of the structure of a model adjustment device provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] In order to better understand the present application, the professional terms involved in the embodiments of the present application are first introduced below.

[0035] The text-image pre-training model based on contrastive learning (also referred to as the contrastive learning model) is a model pre-trained on 400 million pairs of image-text matching pairs, which has a strong generalization ability. By giving the text prompt words corresponding to the contrastive learning model, the model can obtain the similarity between the features of the text prompt words and the features of the corresponding image as the output result of the model. Specifically, the process of the contrastive learning model recognizing the image is to recognize the presentation state of the image for the object, such as whether there is an object in the image (whether there is a cat in the image), or whether there is an object in a certain state in the image (whether there is a white cat in the image), etc. In the recognition process, the contrastive learning model can classify and recognize the image based on the pre-set prompt words. In the above-mentioned text-image pre-training model based on contrastive learning can be a CLIP model, a BLIP (Bootstrapping Language-Image Pre-training, bootstrap language image pre-training) model or a SigLip (Sigmoid Loss for Language-Image Pre-training, language-image pre-training) model, etc., which is not specifically limited in the embodiments of the present application.

[0036] Taking the above contrastive learning model as the CLIP model as an example, when the CLIP model is used to classify an image, the image can be classified by giving the CLIP model two prompt words, "people with tattoos" and "people without tattoos", and letting the CLIP model determine which prompt word has a closer similarity.

[0037] Combine the following Figure 1 This paper introduces the classification process of the CLIP model. Figure 1As shown in FIG. 1 , plane, car, dog, and bird are all target objects (or specified objects) in the embodiment of the present application, and “A photo of a {object}” is the prompt word in the embodiment of the present application. The byte position of object is the byte position occupied by the target object (or specified object) in the prompt word.

[0038] After receiving the prompt word, the Text Encoder of the CLIP model can convert and obtain the text multimodal features T1, T2, T3, ... T corresponding to the prompt word. N That is to say, after receiving the prompt word, the text encoder of the CLIP model extracts the text features of the prompt word and calls its own text multimodal network to convert the extracted text features into text multimodal features.

[0039] After receiving the image, the Image Encoder of the CLIP model can convert and obtain the image multimodal features I1 corresponding to the above image. In other words, after receiving the image, the image encoder of the CLIP model extracts the image features of the image, and calls its own image multimodal network to convert the extracted image features into image multimodal features.

[0040] The CLIP model can then associate the obtained image multimodal feature I1 with the text multimodal features T1, T2, T3, ... T corresponding to the prompt word. N Perform feature similarity matching to obtain the similarities I1·T1, I1·T2, I1·T3, I1·T N Among them, I1·T3 has the highest similarity, and the object in the prompt word corresponding to I1·T3 is dog. Therefore, it can be determined that the object in the image input to the Image Encoder is dog, and the output result of the model is "A photo of a dog".

[0041] Among them, the above-mentioned image multimodal features are the vector features of the image (sample image or test image) in the embodiment of the present application; the above-mentioned text multimodal features are the vector features of the prompt words (sample prompt words) in the embodiment of the present application.

[0042] In order to maintain the network environment of the network platform, it is necessary to review the images or videos uploaded to the platform to avoid the dissemination of images or videos that may cause bad guidance through the platform, for example, the dissemination of images including tattoos, smoke and other objects.

[0043] Exemplarily, a pre-trained CLIP model is used to identify the presentation state of an object in an image. When it is necessary to identify the presentation state of a target object, a sample prompt word and a sample image related to the target object are usually used to perform targeted training on the pre-trained image recognition model. This targeted training enables the trained image recognition model to identify the presentation state of the target object in the input image based on the output result of the image recognition model without the need to input a prompt word in subsequent applications. The output result of the image recognition model is the similarity between the sample prompt word used in model training and the input image, and the similarity between any prompt word and an image represents the similarity between the feature of the prompt word and the feature of the image.

[0044] However, this will cause the feature distribution learned by the image recognition model trained based on the contrastive learning model to shift, that is, the trained image recognition model has a higher recognition accuracy for images with the same business scenarios as the sample images and containing target objects; but has a lower recognition accuracy for images with different business scenarios from the sample images but containing target objects.

[0045] Exemplarily, before training, the model can identify images with tattoos in different business scenarios. However, after training, the images learned by the image recognition model during the training process are all image features of images with tattoos in the same business scenario. In other business scenarios, the business scenarios corresponding to the learned images are different due to the shooting methods used, the architectural styles, landscapes, and characters contained in the images, etc., which cause the learned images to have different business scenarios. This results in a shift in the distribution of features learned by the image recognition model during the training process. For example, the trained image recognition model has a higher recognition accuracy for images with tattoos in farm scenarios, but a lower recognition accuracy for images with tattoos in coffee shop scenarios, resulting in reduced application generalization of the image recognition model.

[0046] Based on this, how to improve the recognition accuracy of the image recognition model for the same object in any business scenario in order to improve the application generalization of the image recognition model has become a technical problem that urgently needs to be solved.

[0047] In order to solve the above technical problems, the embodiments of the present application provide a model adjustment method, device and electronic device.

[0048] Among them, a model adjustment method provided in an embodiment of the present application can be applied to an electronic device, and the electronic device is specifically used to train and adjust the model. In a specific application, the electronic device can be a smart phone, a tablet computer, etc., which is all reasonable.

[0049] A model adjustment method provided in an embodiment of the present application may include the following steps:

[0050] Determine a first model parameter to be adjusted of a target image recognition model and a second model parameter of a pre-trained contrastive learning model; wherein the target image recognition model is an image recognition model for identifying a target object obtained by training the contrastive learning model based on sample images in a first business scenario; the first model parameter and the second model parameter are parameters having the same function in an image recognition process;

[0051] Acquire multiple weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain a target model parameter corresponding to the weight parameter group; wherein each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter; and the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value;

[0052] For each target model parameter, using the target model parameter to replace the first model parameter of the target image recognition model, to obtain each candidate image recognition model;

[0053] For each candidate image recognition model, a test image is input into the candidate image recognition model to obtain an output result corresponding to the test image, and based on the output result corresponding to the test image and a label of the test image, an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model is determined; wherein the test image is an image in a second business scenario and is provided with a label, and the label is used to characterize the presentation state of the target object;

[0054] A candidate image recognition model with the largest evaluation value among the candidate image recognition models is selected as a target image recognition model matching the second business scenario.

[0055] From the above, it can be seen that the solution provided by the embodiment of the present application can, when adjusting the model, fuse the model parameters of the target image recognition model and the model parameters of the corresponding comparative learning model according to the weight parameter group, so as to provide the target image recognition model with the data features required for image recognition of the same object in the second business scenario through parameter fusion, thereby improving the recognition accuracy of the image recognition model for the same object in any business scenario, so as to improve the application generalization of the image recognition model.

[0056] Furthermore, the adjusted target image recognition model is a candidate image recognition model with the largest evaluation value determined based on the output results of the model and the label of the test image. That is to say, the adjusted target image recognition model is the recognition model with the highest recognition accuracy among all candidate recognition models when performing image recognition on images containing target objects in the second business scenario. Therefore, applying a model adjustment method provided by an embodiment of the present application can improve the accuracy of the adjusted target image recognition model when performing recognition on images containing target objects in the second business scenario while ensuring the recognition accuracy of the adjusted target image recognition model for images containing target objects in the first business scenario.

[0057] A model adjustment method provided in an embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0058] Figure 2 A flow chart of a model adjustment method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the method may include the following steps

[0059] S201: Determine a first model parameter to be adjusted for a target image recognition model and a second model parameter for a pre-trained contrastive learning model;

[0060] Among them, the target image recognition model is an image recognition model for identifying the target object obtained by training the contrast learning model based on the sample images in the first business scenario; the first model parameters and the second model parameters are parameters with the same function in the image recognition process.

[0061] In this application, the contrast learning model is pre-trained using sample images in the first business scenario to obtain an image recognition model for identifying the target object as the target image recognition model to be adjusted. For the convenience of writing, the specific implementation method for training the target image recognition model is detailed in steps S401-S405 below, which will not be repeated here.

[0062] In this way, when adjusting the model, the first model parameters of the target image recognition model and the second model parameters of the comparative learning model can be determined.

[0063] In the image recognition process, it is necessary to determine the vector features of the prompt word and the vector features of the input image, so as to complete the recognition of the image by determining the similarity of the vector features of the two. It can be seen that the model parameters that need to be adjusted are the functional parameters involved in the image recognition process, for example, the parameters used to provide the vector features of the prompt word for the recognition process, or the parameters used to provide the vector features of the image for the recognition process.

[0064] Moreover, when performing parameter fusion, it is necessary to fuse parameters with the same functions accordingly, that is, to fuse the parameters of the vector features of the target image recognition model used to provide prompt words for the recognition process with the parameters of the vector features of the contrastive learning model used to provide prompt words for the recognition process, or to fuse the parameters of the vector features of the target image recognition model used to provide images for the recognition process with the parameters of the vector features of the contrastive learning model used to provide images for the recognition process, so as to reduce the possibility of parameter errors in the image recognition model after parameter fusion.

[0065] Therefore, the determined first model parameter and the determined second model parameter are parameters having the same function in the image recognition process.

[0066] Optionally, in one implementation, Figure 3 As shown, it is a flow chart of a model adjustment method provided by an embodiment of the present application. When the target image recognition model includes a fully connected layer and an image encoder in a contrastive learning model, in a model adjustment method provided by an embodiment of the present application, the above step S201, determining the first model parameter to be adjusted of the target image recognition model and the second model parameter of the pre-trained contrastive learning model, may include the following steps:

[0067] S2011: Determine a weight parameter of a fully connected layer of a target image recognition model and a first encoder parameter of an image encoder of the target image recognition model as a first model parameter of the target image recognition model;

[0068] S2012: Determine a vector feature group generated based on a pre-trained contrastive learning model and a second encoder parameter of an image encoder of the contrastive learning model as a second model parameter of the contrastive learning model;

[0069] The vector feature group is obtained by concatenating the vector features of each sample prompt word generated by the text encoder in the contrastive learning model.

[0070] In this implementation, the model structure of the trained target image recognition model includes: a fully connected layer and an image encoder in a contrastive learning model, wherein the weight parameters of the fully connected layer are used to provide vector features of sample prompt words for the image recognition process. Therefore, during the application of the target image recognition model, there is no need to input prompt words, only the image to be recognized.

[0071] Optionally, the weight parameters of the fully connected layer are weight parameters when the target image recognition model is trained based on the initial weight parameters of the fully connected layer, wherein the initial weight parameters of the fully connected layer are predetermined by the vector features of each sample prompt word.

[0072] In the training process of the above-mentioned target image recognition model, the sample prompt words used for training are image description texts with the presentation state of the target object as the description content, and the sample images used for training are sample images containing the target object in the first business scenario. This also results in that the target image recognition model can only guarantee the recognition accuracy rate when performing image recognition on images containing the target object in the first business scenario, while the recognition accuracy rate is lower when performing recognition on images containing the target object in other business scenarios. For example, when the target object is cigarettes, the first business scenario is a coffee shop, and the second business scenario is a farm, the trained target image recognition model can only guarantee the recognition accuracy rate when performing image recognition on images containing cigarettes in the coffee shop scenario, while for images containing cigarettes in the farm scenario, since the architectural style of the farm is completely different from that of the coffee shop, the recognition accuracy rate may be lower during recognition.

[0073] Therefore, in this implementation, by determining the first model parameters and the second model parameters, a data basis is provided for subsequent parameter fusion. When determining the model parameters, the weight parameters of the fully connected layer of the target image recognition model and the first encoder parameters of the image encoder of the target image recognition model are used as the first model parameters of the target image recognition model.

[0074] In order to improve the accuracy of the target model parameters obtained after parameter fusion, that is, using each weight parameter group, the first model parameters and the second model parameters are weighted and summed to obtain the accuracy of each target model parameter. For the different parts (positions) contained in the target image recognition model, the model parameters corresponding to the part can be determined, and the parameters of the part in the two models can be used for parameter fusion.

[0075] However, the contrastive learning model corresponding to the target image recognition model does not include a fully connected layer. Considering that the initial weight parameters of the fully connected layer are predetermined based on the vector features of multiple sample prompt words, that is, the role of the fully connected layer in the target image recognition model is to provide the vector features of the prompt words for image recognition. Therefore, the vector features of each prompt word generated by the text encoder in the contrastive learning model can be spliced ​​to obtain the vector feature group of the contrastive learning model. The determined vector feature group and the second encoder parameters of the image encoder of the contrastive learning model are used as the second model parameters.

[0076] Among them, since the purpose of this application is to improve the recognition accuracy of the image recognition model for the same object in any business scenario, the vector features generated by the contrastive learning model are the vector features corresponding to the sample prompt words of the image description text with the presentation state of the target object as the description content.

[0077] In this implementation, the weight parameters and the first encoder parameters of the fully connected layer of the target image recognition model are respectively determined as the first model parameters of the target image recognition model; and the vector feature group and the second encoder parameters of the contrastive learning model are determined as the second model parameters of the contrastive learning model, so that in subsequent parameter fusion, the model parameters of the corresponding parts in the model can be targeted for fusion to improve the accuracy of the determined target model parameters.

[0078] S202: Acquire multiple weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain the target model parameter corresponding to the weight parameter group.

[0079] Each weight parameter group includes a first weight assigned to a first model parameter and a second weight assigned to a second model parameter; the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value.

[0080] In the present application, in order to improve the objectivity of the obtained target model parameters, multiple weight parameter groups can be obtained, wherein each weight parameter group includes a first weight assigned to a first model parameter and a second weight assigned to a second model parameter. Afterwards, for each weight parameter group, the determined first model parameter and second model parameter are weighted and summed according to the weight parameter group to obtain the target model parameters corresponding to the weight parameter group.

[0081] Optionally, multiple weights are obtained from a predetermined threshold range at predetermined intervals, and the determined multiple weights are combined into multiple weight parameter groups according to a predetermined combination condition; wherein the predetermined combination condition is that a weight ratio group contains a specified number of weights, and the sum of the specified number of weights is a predetermined sum value. This application does not specifically limit the specified number and the predetermined sum value.

[0082] For example, between the predetermined threshold range [0, 1], with 0.1 as the predetermined interval, select weights 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 and 1. Among them, the predetermined combination condition is 2 weights, and the sum of the 2 weights is 1. Therefore, the multiple weight parameter groups obtained are: weight parameter group 1 includes a first weight 0 and a second weight 1, weight parameter group 2 includes a first weight 0.1 and a second weight 0.9, weight parameter group 3 includes a first weight 0.2 and a second weight 0.8, and so on, until it is determined that weight parameter group 10 includes a first weight 0 and a second weight 1. Therefore, using each of the 10 weight parameter groups determined, the first model parameter and the second model parameter are weighted and summed to obtain each target model parameter.

[0083] Optionally, in one implementation, the target model parameters include target weight parameters and target encoder parameters; the above step S202, for each weight parameter group, performs weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain the target model parameter corresponding to the weight parameter group, may include the following steps:

[0084] Step A: For each weight parameter group, the first weight in the group is used as the weighted weight of the weight parameter of the fully connected layer and the second weight in the group is used as the weighted weight of the vector feature group, and the weight parameters and the vector feature group are weighted summed to obtain the target weight parameters; the first weight in the group is used as the weighted weight of the first encoder parameter and the second weight in the group is used as the weighted weight of the second encoder parameter, and the first encoder parameter and the second encoder parameter are weighted summed to obtain the target encoder parameter.

[0085] In this implementation, in order to improve the accuracy of the determined target model parameters, the model parameters corresponding to the different parts included in the target image recognition model can be determined, and the parameters of the two models that have the same function in the image recognition process can be fused. That is, for the target image recognition model including the image encoder and the fully connected layer, the target model parameters to be determined include the target encoder parameters for the image encoder, and the target weight parameters for the fully connected layer, that is, the target weight parameters for the vector features of the prompt word.

[0086] Moreover, the parameter fusion in the present implementation method is to fuse the model parameters of the first model parameter of the target image recognition model and the second model parameter of the comparative learning model. Therefore, the number of weights included in each determined weight parameter group is 2, that is, each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter.

[0087] Thus, for each weight parameter group, the first weight in the group is used as the weighted weight of the weight parameter of the fully connected layer, and the second weight in the group is used as the weighted weight of the vector feature group, and the weight parameters of the fully connected layer and the vector features are weighted summed to obtain the target weight parameters. In addition, the first weight in the group is used as the weighted weight of the first encoder parameter, and the second weight in the group is used as the weighted weight of the second encoder parameter, and the first encoder parameter and the second encoder parameter are weighted summed to obtain the target encoder parameter.

[0088] In this implementation, the target weight parameters for the vector features of the prompt word are obtained according to the corresponding weight calculation, which are used as the weight parameters to which the fully connected layer in the target image recognition model needs to be adjusted after parameter fusion; the target encoder parameters for the image encoder are obtained according to the corresponding weight calculation, which are used as the encoder parameters to which the image encoder in the target image recognition model needs to be adjusted after parameter fusion, thereby achieving targeted fusion of model parameters for corresponding parts in the target image recognition model to improve the accuracy of the determined target model parameters.

[0089] S203: For each target model parameter, use the target model parameter to replace the first model parameter of the target image recognition model to obtain each candidate image recognition model.

[0090] The parameter fusion provided in this application is performed on the basis of the target image recognition model, that is, the model structure of the adjusted target image recognition model is the same as the target image recognition model, but the model parameters are different. Therefore, after determining each target model parameter, each target model parameter can be used to replace the first model parameter of the target image recognition model to obtain each candidate image recognition model.

[0091] S204: For each candidate image recognition model, input the test image into the candidate image recognition model to obtain an output result corresponding to the test image, and determine an evaluation value corresponding to the candidate image recognition model for characterizing the recognition accuracy of the model based on the output result corresponding to the test image and a label of the test image;

[0092] The test image is an image in the second business scenario and is provided with a label, wherein the label is used to characterize the presentation state of the target object.

[0093] In the present application, an image belonging to the second business scenario is determined as a test image for testing each candidate image recognition model, and the test image is provided with a label for characterizing the presentation state of the target object. Then, the first model parameters are replaced with each target model parameter obtained after parameter fusion, so that the obtained candidate image recognition model can learn the feature data about the target object in the second business scenario. Specifically:

[0094] The vector features of the sample prompt words represented by the weight parameters of the fully connected layer and the vector features of multiple sample prompt words generated by the contrast learning model are fused according to the corresponding weights to obtain the target weight parameters. Among them, since the weight parameters of the fully connected layer are adjusted based on the initial weight parameters spliced ​​based on the vector features of each sample prompt word, and the sample data used for the adjustment is under the first business scenario, the vector features of each sample prompt word represented by the weight parameters of the fully connected layer are more biased towards the first business scenario, but the vector features of the sample prompt words directly generated are more biased towards the second business scenario, and the data biases of the two regarding the text features of the target object are different.

[0095] Therefore, the target weight parameters obtained through parameter fusion can provide the candidate image recognition model with text features of the target objects in the first business scenario and the second business scenario at the same time; the target encoder parameters obtained can provide the candidate image recognition model with image features of the target objects in the first business scenario and the second business scenario at the same time.

[0096] In order to test the recognition accuracy of each determined candidate image recognition model for an image containing a target object in the second business scenario, for each candidate image recognition model, the acquired test image is input into the candidate image recognition model to obtain an output result corresponding to the test image, which is used to characterize the similarity between the vector features of each sample prompt word and the vector features of the test image. Then, based on the output result corresponding to the test image and the label set for the test image itself, an evaluation value corresponding to the candidate image recognition model for characterizing the recognition accuracy of the model is determined.

[0097] Among them, the presentation state of the above-mentioned target object can be understood as whether there is a target object in the test image, for example, whether there is smoke in the image, or whether there is a cat in the image, etc.; it can also be understood as whether there is a target object in a certain state or belonging to a certain attribute in the test image, for example, whether there is smoke in a burning state in the image, or whether there is a white cat in the image, etc., which are all reasonable.

[0098] It should be noted that the label used to characterize the presentation state of the target object can be a digital label or a text label, and the embodiments of the present application do not specifically limit this. For example, if the target object exists in the test image, the label of the test image can be set to 1; if the target object does not exist in the test image, the label of the test image can be set to 0.

[0099] Optionally, based on the output result corresponding to the test image and the label of the test image, the loss value of the candidate image recognition model is determined as an evaluation value for characterizing the recognition accuracy of the model; wherein, the smaller the determined loss value, the larger the corresponding evaluation value, and accordingly, the higher the recognition accuracy of the candidate image recognition model for the image of the target object in the second business scenario.

[0100] Optionally, multiple evaluation values ​​are obtained, and an average value of the multiple evaluation values ​​is used as the final evaluation value to improve the objectivity and universality of the determined evaluation value.

[0101] S205: Selecting a candidate image recognition model with the largest evaluation value among the candidate image recognition models as a target image recognition model that matches the second business scenario.

[0102] In the present application, since the determined evaluation value is used to characterize the recognition accuracy of the candidate image recognition model, that is to say, the larger the determined evaluation value is, the higher the recognition accuracy of the candidate image recognition model for the image of the target object in the second business scenario is. Therefore, after determining the evaluation value of each candidate image recognition model, the candidate image recognition model with the largest evaluation value can be determined as the target image recognition model that matches the second business scenario, thereby ensuring the recognition accuracy of the adjusted target image recognition model for the image containing the target object in the first business scenario, while improving the recognition accuracy of the adjusted target image recognition model for the image containing the target object in the second business scenario.

[0103] From the above, it can be seen that the solution provided by the embodiment of the present application can, when adjusting the model, fuse the model parameters of the target image recognition model and the model parameters of the corresponding comparative learning model according to the weight parameter group, so as to provide the target image recognition model with the data features required for image recognition of the same object in the second business scenario through parameter fusion, thereby improving the recognition accuracy of the image recognition model for the same object in any business scenario, so as to improve the application generalization of the image recognition model.

[0104] Furthermore, the adjusted target image recognition model is a candidate image recognition model with the largest evaluation value determined based on the output results of the model and the label of the test image. That is to say, the adjusted target image recognition model is the recognition model with the highest recognition accuracy among all candidate recognition models when performing image recognition on images containing target objects in the second business scenario. Therefore, applying a model adjustment method provided in an embodiment of the present application can improve the accuracy of the adjusted target image recognition model when performing recognition on images containing target objects in the second business scenario while ensuring the recognition accuracy of the adjusted target image recognition model for images containing target objects in the first business scenario.

[0105] Optionally, in one implementation, the output result corresponding to the test image includes the similarity between the vector feature of a sample prompt word and the vector feature of the test image. The above step S204, based on the output result corresponding to the test image and the label of the test image, determines the evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model, which may include the following steps:

[0106] Step B1: determining the confidence level corresponding to each sample prompt word based on the similarity between the vector feature of each sample prompt word and the vector feature of the test image;

[0107] The confidence level corresponding to any sample prompt word is used to characterize the presentation state described in the sample prompt word, which is the predicted probability of the presentation state of the target object in the test image.

[0108] Step B2: Based on the confidence level corresponding to each sample prompt word and the label of the test image, determine the evaluation value corresponding to the candidate image recognition model, which is used to characterize the recognition accuracy of the model.

[0109] In this implementation, for each candidate image recognition model, after the test image is input into the candidate image recognition model, the similarity between the vector features of each sample prompt word and the vector features of the test image can be obtained as the output result corresponding to the test image. Then, based on each similarity, the confidence corresponding to each sample prompt word is determined, thereby determining the presentation state described by each sample prompt word, and predicting the probability of the presentation state of the target object in the test image. In other words, the higher the confidence corresponding to a determined sample prompt word, the higher the probability that the presentation state described by the sample prompt word is the presentation state of the target object in the test image.

[0110] Among them, the label set for the test image represents the actual state presented by the target object in the test image. Therefore, after determining the confidence corresponding to each sample prompt word, based on the above confidence and the label of the test image, the evaluation value corresponding to the candidate image recognition model can be determined, which is used to characterize the model recognition accuracy.

[0111] In this implementation, the confidence level is used to digitize the presentation state described in each sample prompt word as the probability of the presentation state of the target object in the test image, so as to match the test image with the corresponding sample prompt word, thereby improving the accuracy of the analysis of the presentation state of the test image for the target object, and further improving the accuracy of the determined evaluation value.

[0112] Optionally, in one implementation, the above step B1, determining the confidence level corresponding to each sample prompt word based on the similarity between the vector feature of each sample prompt word and the vector feature of the test image, may include the following steps:

[0113] Step B11: normalize the similarity between the vector feature of each sample prompt word and the vector feature of the test image to obtain the confidence level corresponding to the sample prompt word.

[0114] In this implementation, the similarity between the vector feature of each sample prompt word and the vector feature of the test image is normalized to obtain the confidence level corresponding to the sample prompt word.

[0115] Optionally, for each sample prompt word, a predetermined formula is used to calculate the confidence level corresponding to the sample prompt word; wherein the predetermined formula includes:

[0116]

[0117] σ(z i ) represents the confidence corresponding to the i-th sample prompt word of the target object, K represents the total number of sample prompt words of the target object, z i Characterize the feature score corresponding to the i-th sample prompt word, z j Characterize the feature score corresponding to the j-th sample prompt word.

[0118] The feature score corresponding to each sample prompt word is used to characterize the similarity between the sample prompt word and the test image, that is, the similarity between the vector feature of each sample prompt word and the vector feature of the test image in the embodiment of the present application.

[0119] In this implementation, by normalizing the similarity between the vector features of each sample prompt word and the vector features of the test image to a specific interval, for example, [0, 1] or [-1, 1], etc., the data distribution is made statistically significant, and the probability distribution of the similarity is presented more statistically and intuitively, thereby improving the comparability of the similarity corresponding to each sample prompt word.

[0120] Optionally, in one embodiment, Figure 4 A training process diagram of a target image recognition model provided in an embodiment of the present application, the training process may include the following steps:

[0121] S401: Acquire a sample image;

[0122] The sample image is provided with a label for characterizing the presentation state of the target object.

[0123] In this embodiment, since the public data set may not contain training data related to actual business scenarios, for example, the public data set does not contain training data related to tattoos (sample images and sample prompt words). Therefore, the image recognition model trained using the public data set has a low recognition accuracy rate for objects contained in images in highly targeted, actual business scenarios. Taking the target object as a tattoo as an example, if there are no tattoo-related images and prompt words in the public data set, the image recognition model trained using the public data set has a low recognition accuracy rate for whether there is a tattoo in the image.

[0124] Therefore, an image provided with a label characterizing the presentation state of the target object can be acquired as a sample image.

[0125] It is understandable that the number of sample images acquired may be one or more, and the embodiment of the present application does not specifically limit this.

[0126] Among them, the presentation state of the above-mentioned sample object can be understood as whether the sample object exists in the sample image, for example, whether there is smoke in the image, or whether there is a cat in the image, etc.; it can also be understood as whether there is a sample object in a certain state or belonging to a certain attribute in the sample image, for example, whether there is smoke in a burning state in the image, or whether there is a white cat in the image, etc., which are all reasonable.

[0127] It should be noted that the label used to characterize the presentation state of the target object can be a digital label or a text label, and the present application embodiment does not specifically limit this. For example, if the target object exists in the sample image, the label of the sample image can be set to 1; if the target object does not exist in the sample image, the label of the sample image can be set to 0.

[0128] S402: Input the sample image into the image recognition model to be trained to obtain the output result corresponding to each sample prompt word;

[0129] Among them, the image recognition model includes a fully connected layer with predetermined initial weight parameters and an image encoder in a contrastive learning model; the output result corresponding to each sample prompt word is used to characterize the similarity between the vector feature of the sample prompt word and the vector feature of the sample image.

[0130] In this embodiment, it is considered that in the prior art, a contrastive learning model is usually used as an image recognition model for image recognition. When performing image recognition, this type of model calls a text encoder and an image encoder to process the input prompt word and image respectively, that is, calls the text encoder to process the input prompt word to obtain the vector features of the prompt word, and calls the image encoder to process the input image to obtain the vector features of the image. In this way, this type of model can recognize the image by determining the similarity between the vector features of the prompt word and the vector features of the image.

[0131] That is to say, the key point of the image recognition model to recognize the image is to obtain the vector features of the prompt word and the vector features of the image, and to determine the similarity of the two vector features obtained. At this time, if the image recognition model is trained for a certain type of object or a certain type of practical application scenario, then it can be understood that the prompt word related to a certain type of object or a certain type of practical application scenario can be fixed, and then, the vector features of the predetermined dimension of the generated prompt word are also fixed within a certain range.

[0132] Furthermore, considering that the role of the fully connected layer in the entire image recognition model is to realize the processing of multimodal data by calculating the similarity of multimodal data; and the process of processing the vector features in the embodiment of the present application is: using the vector features of the sample image and the vector features of each sample prompt word, calculate the similarity of each sample prompt word to the sample image. It can be seen that the processing process of the fully connected layer for multimodal data is the same as the processing process for vector features in the embodiment of the present application. Therefore, when the vector features of the predetermined dimensions of the generated sample prompt words are fixed within a certain range, the parameters determined based on the vector features of multiple sample prompt words can be used as the weight parameters of the fully connected layer itself, thereby using the fully connected layer to replace the text encoder in the image recognition model, and providing the image recognition process with vector features about the sample prompt words through the weight parameters of the fully connected layer.

[0133] Therefore, the fully connected layer can be spliced ​​after the image encoder through data association to achieve the reconstruction of the internal structure of the image recognition model and obtain the image recognition model to be trained. In addition, before the model training starts, each image description text with the presentation state of the sample object as the description content is used as each sample prompt word, and then the vector feature of the predetermined dimension of each sample prompt word is obtained, and the matrix determined based on the vector features of multiple sample prompt words is used as the initial weight parameter of the fully connected layer to provide the vector features of the sample prompt words for subsequent data processing.

[0134] That is, the image recognition model to be trained includes a fully connected layer with predetermined initial weight parameters and an image encoder in a pre-trained contrastive learning model.

[0135] In this way, after obtaining the sample image, the sample image can be input into the image recognition model to be trained to obtain the similarity between the vector feature of each sample prompt word and the vector feature of the sample image as the output result corresponding to the sample prompt word.

[0136] Among them, the specific implementation method for determining the initial weight parameters of the fully connected layer is detailed in step D below and will not be repeated here.

[0137] It should be noted that the feature score can be used to represent the output result corresponding to each sample prompt word, that is, the feature score can be used to represent the similarity between the vector feature of the sample prompt word and the vector feature of the sample image. This embodiment of the application does not make any specific limitation on this.

[0138] Optionally, in one implementation, the above step S402, inputting the sample image into the image recognition model to be trained to obtain the output result corresponding to each sample prompt word, may include the following steps:

[0139] Step C: Input the sample image into the image encoder in the image recognition model to be trained so that the image encoder generates vector features of the sample image, and transmits the vector features of the sample image to the fully connected layer to process the vector features of the sample image based on the current weight parameters of the fully connected layer itself to obtain the output result corresponding to each sample prompt word.

[0140] In this implementation, after obtaining the sample image, the sample image is input into the above-mentioned image recognition model to be trained, so that the image encoder in the image recognition model generates vector features of the sample image, and the vector features of the sample image are transmitted to the fully connected layer in the image recognition model, so as to process the vector features of the sample image based on the current weight parameters of the fully connected layer itself, that is, calculate the similarity between the modal vector features of each sample prompt word and the vector features of the sample image, thereby obtaining the output result corresponding to each sample prompt word.

[0141] Optionally, cosine similarity is used to calculate the distance between the vector feature of each sample prompt word and the modulus vector feature of the sample image; specifically, the closer the distance between the vector feature of the sample prompt word and the vector feature of the sample image, the higher the similarity between the two vector features, and conversely, the farther the distance between the vector feature of the sample prompt word and the vector feature of the sample image, the lower the similarity between the two vector features.

[0142] Optionally, considering different data categories, such as text data, image data, etc., whose corresponding vector features are in different dimensions, in order to facilitate the subsequent determination of the similarity between the vector features of each sample prompt word and the vector features of the sample image, the vector features corresponding to data of different data types can be unified into the same dimension. Therefore, after obtaining the sample image, the image encoder of the image recognition model extracts the image features of the sample image, and generates vector features of a predetermined dimension of the sample image based on the extracted image features. Among them, the vector features of the image features can be represented by the feature matrix, and this is not specifically limited in the embodiments of the present application.

[0143] In addition, considering that the image sizes of the acquired sample images are different, in order to ensure that each acquired sample image can be converted into image features of the same dimension, so as to be further converted into vector features of a predetermined dimension, the sample image can be scaled to a predetermined size before generating the vector features of the sample image, so as to be used in the process of generating the vector features. Based on this, optionally, after acquiring the sample image, the sample image is scaled to a predetermined size, and after image features are extracted from the sample image of the predetermined size, the extracted image features are converted into vector features of a predetermined dimension.

[0144] Exemplarily, after acquiring a sample image, the image size of the sample image is scaled to a fixed size, for example, 224×224, or 336×336, and then the scaled sample image is input into a module for generating image vector features to generate vector features of the sample image.

[0145] In this implementation, the dimensions of the image features of the sample image are converted to obtain vector features of the sample image at a predetermined dimension, thereby facilitating subsequent determination of the similarity between the vector features of each sample prompt word and the vector features of the sample image.

[0146] It should be noted that the embodiments of the present application involve a text-image pre-training model based on contrastive learning, and the main ability of this type of model is to align image features and text features in the same feature space (dimension). Therefore, in theory, only the trained model layer in the pre-training model can be used to convert image or text features to vector features, that is, the image vector features are converted through the image encoder in the pre-training model, and the text vector features are converted through the text encoder in the pre-training model. Other untrained methods cannot be used, otherwise, the obtained vector features of the image and the vector features of the text will not be in the same feature space from a mathematical perspective.

[0147] Based on this, optionally, the sample image is input into an image encoder of a pre-constructed image recognition model to be trained, so that the image encoder extracts image features of the sample image and converts the extracted image features to obtain vector features of the sample image; each sample prompt word is input into a text encoder of a pre-trained contrastive learning model corresponding to the image recognition model before construction, so that the text encoder extracts text features of the sample prompt word and converts the extracted text features to obtain vector features of the sample prompt word, and then, the obtained vector features of the sample prompt word are used as matrix elements contained in the initial weight parameters of the fully connected layer.

[0148] In this way, since the generation method of the vector features of the sample image and the vector features of the sample prompt words is pre-trained, the obtained vector features of the sample image and the vector features of the sample prompt words are in the same feature space from a mathematical perspective, that is, the vector features obtained using the image encoder of the above-mentioned image recognition model and the text encoder of the contrastive learning model are in the same predetermined dimension.

[0149] S403: Based on the output result corresponding to each sample prompt word and the label of the sample image, determine the loss value corresponding to the image recognition model.

[0150] In this embodiment, since the sample image is provided with a label for representing the presentation state of the target object, that is, the presentation state of the target object represented by the label of the sample image is the actual state presented by the target object in the sample image. Therefore, after obtaining the output result corresponding to each sample prompt word, the loss value corresponding to the image recognition model can be determined through the above output result and the label of the sample image.

[0151] S404: When it is determined based on the loss value that the image recognition model does not meet the model convergence condition, the current weight parameters of the fully connected layer and the image encoder are adjusted based on the loss value, and the process returns to the above step S401;

[0152] S405: When it is determined based on the loss value that the image recognition model meets the model convergence condition, the model training is terminated to obtain the target image recognition model.

[0153] In this embodiment, after determining the loss value corresponding to the image recognition model, it is possible to further determine whether the image recognition model meets the model convergence conditions, thereby determining whether to continue model training.

[0154] When it is determined based on the loss value that the image recognition model does not meet the model convergence conditions, the current weight parameters of the fully connected layer and the image encoder can be adjusted based on the loss value, and the process returns to step S401 to obtain a sample image, and based on the next sample image obtained, the process executes steps S401-S405 to train the image recognition model.

[0155] When it is determined based on the loss value that the image recognition model meets the model convergence condition, the model training is terminated to obtain a target image recognition model for identifying the target object.

[0156] Optionally, in order to improve the recognition accuracy of the target image recognition model for the image, the above-mentioned model convergence condition may be that the loss value reaches a predetermined loss value, that is, when the determined loss value reaches the predetermined loss value, the training of the model is terminated.

[0157] Optionally, in order to improve the training efficiency of the model training, the model training end condition can be set to the model training times reaching a predetermined number of times, that is, when the model training times reaches the predetermined number of times, the model training is ended.

[0158] Optionally, a back propagation algorithm is used to calculate the gradient value of the feature vector corresponding to the current weight parameter of the fully connected layer and the encoder parameter of the image encoder based on the loss value, and the calculated gradient value is used to adjust the current weight parameter of the fully connected layer and the encoder parameter of the image encoder. Thus, in the process of model training, the model parameters are adjusted according to the determined loss value to balance the above loss value and optimize the model performance.

[0159] It should be noted that the electronic device used to execute the above steps S401-S405 can be the same electronic device as the electronic device executing the above steps S201-S205, or can be a different electronic device, and the embodiment of the present application does not make any specific limitation.

[0160] Optionally, in one embodiment, the method for determining the initial weight parameters of the fully connected layer may include the following steps:

[0161] Step D: Obtain the vector features of each sample prompt word, and concatenate the vector features of each sample prompt word as a row of matrix elements to obtain the initial weight parameters of the fully connected layer.

[0162] In this embodiment, after obtaining each sample prompt word, the vector features of the sample prompt word can continue to be obtained, so that the vector features of each sample prompt word are spliced ​​in a manner of splicing the vector features of each sample prompt word as a row of matrix elements to obtain the initial weight parameters of the fully connected layer.

[0163] Optionally, an image description text with the presentation status of the target object as the description content is obtained in advance as a sample prompt word; wherein the above-mentioned sample prompt word can be a single word or a long sentence composed of multiple words, and the embodiment of the present application does not make any specific limitation on this.

[0164] In this way, after obtaining each sample prompt word, the text feature of each sample prompt word can be extracted. Considering different data types, such as text data, image data, etc., the corresponding vector features are in different dimensions. In order to facilitate the subsequent determination of the similarity between the vector features of each sample prompt word and the vector features of the sample image, the vector features corresponding to data of different data types can be unified into the same dimension. Therefore, after obtaining the text feature of each sample prompt word, the vector feature of the predetermined dimension of the sample prompt word can be generated.

[0165] Optionally, the feature matrix is ​​used to represent the vector features of the text features, that is, the feature matrix is ​​used to represent the vector features of the sample prompt words. Therefore, after obtaining the vector features of multiple sample prompt words, the vector features of the multiple sample prompt words can be spliced ​​into a matrix to obtain the initial weight parameters of the fully connected layer.

[0166] Optionally, a target keyword and other non-keywords that can be combined with the target keyword are set in advance for a presentation state of the target object. In this way, through the combination of a target keyword and other non-keywords, an image description text with the presentation state of the target object as the description content can be obtained as a sample prompt word, thereby obtaining multiple sample prompt words.

[0167] The combination of the above target keywords and other non-keywords may include:

[0168] (1) For different presentation states of the same target object, corresponding target keywords are set, and for each target keyword, other non-keywords are randomly combined with the target keyword to obtain multiple sample prompt words. For example, when the target object is a tattoo, the presentation state of the target object may include: the presence of a tattoo and the absence of a tattoo. In this case, the target keywords that can be set are the presence of a tattoo and the absence of a tattoo.

[0169] (2) For a presentation state of the same target object, a corresponding target keyword is set, and multiple sample prompt words are obtained by randomly combining the target keyword with other non-keywords. For example, when the target object is a tattoo, the presentation state of the target object may include: the presence of a tattoo and the absence of a tattoo. In this case, the target keyword that can be set is one of the presence of a tattoo and the absence of a tattoo.

[0170] The embodiment of the present application does not limit the number of target keywords in the multiple sample prompt words, nor the number of other non-keywords.

[0171] In this embodiment, before training the image recognition model to be trained, the initial weight parameters of the fully connected layer are determined in advance to provide pre-training parameters for the fully connected layer for the training process of the image recognition model. The pre-training parameters enable the fully connected layer to have the ability to classify images at the beginning of model training. Therefore, in the process of training the image recognition model, only a small number of sample images are needed to enable the trained target image recognition model to obtain better training results.

[0172] Moreover, by adjusting the current weight parameters of the fully connected layer and the parameters of the image encoder in a reconstructed image recognition model, the image recognition model can be trained. There is no need to train the text encoder and image encoder of the comparative learning model in the prior art, thereby reducing the number of encoders that need to be trained, thereby improving the efficiency of model training.

[0173] In addition, when training the contrastive learning model in the prior art, it is required that each sample image has a unique sample prompt word. However, in actual application, each sample image may only have two sample prompt words, with tattoos or without tattoos. That is to say, the sample prompt words in the actual application process do not meet the training conditions of the contrastive learning model, resulting in the inability to train the contrastive learning model. Therefore, based on the structure of the contrastive learning model, a fully connected layer with initial weight parameters is used to replace the original text encoder in the contrastive learning model to achieve reconstruction of the model structure and obtain an image recognition model. In the training process of the image recognition model, the initial weight parameters of the fully connected layer are used to provide vector features about the sample prompt words for the entire model training process, so there is no need to consider whether the sample prompt words in the actual application process meet the training conditions of the contrastive learning model.

[0174] A model adjustment method provided in an embodiment of the present application is specifically introduced below in conjunction with a specific embodiment.

[0175] Since conventional deep learning recognition models need to label large quantities of data, and in actual business, the data that needs to be recognized has a long-tail distribution, that is, the data that needs to be recognized accounts for a small proportion of the total data and is difficult to collect, resulting in high data labeling costs and slow model iteration cycles. Therefore, considering that the CLIP model does not need to collect data to complete image recognition, comparative learning models such as the CLIP model can be used to perform zero-sample predictions to achieve automatic recognition of images. However, comparative learning models such as the CLIP model are trained based on pre-training data obtained from public data sets, and the above public data sets may not contain business data related to actual scenarios. For example, the public data set does not contain the features required for identifying tattoos, resulting in the trained model having a low recognition accuracy for images with tattoos during application in actual scenarios.

[0176] Moreover, when training contrastive learning models such as the CLIP model, the text encoder and image encoder contained in the contrastive learning model need to be trained simultaneously, during which the number of required training samples and the training time are greatly increased, resulting in reduced model training efficiency.

[0177] For example, Figure 5 FIG. 1 is a flowchart of an existing training method for a contrastive learning model. Figure 5 As shown, in the training process of the existing contrastive learning model, it is necessary to obtain the key prompt words with airplane, butterfly and pizza as the objects respectively, combine the key prompt words with other non-keywords, and then output each prompt word obtained after the above combination to the Text Encoder of the CLIP model to obtain the Text features corresponding to each prompt word; then, input the image into the Image Encoder of the CLIP model to obtain the Image features corresponding to the image; then, calculate the Similarity scores of the text features and the image features as the similarity between the two, and thus, based on the determined similarity, calculate the loss value of the Image Encoder and the loss value of the Text Encoder. By Figure 5 It can be seen that when training the existing contrastive learning model, the text encoder and the image encoder need to be trained at the same time, which makes the training efficiency of the model low.

[0178] Considering the model structure of the conventional deep learning recognition model, exemplary, Figure 6It is a schematic diagram of the structure of a conventional deep learning recognition model. In the recognition process of the deep learning recognition model, after the image is input into the deep learning recognition model, a feature extraction backbone network (the feature extraction backbone network may include convolution layer 1, pooling layer 1, convolution layer 2 and pooling layer 2) is used to extract the image features of the image, and then one to three fully connected layers (fully connected layer 1, fully connected layer 2 and fully connected layer 3) are used to calculate the logistic regression score of the image features to the specific label, and then a softmax function is used to summarize the logistic regression scores of all labels to obtain the confidence of each label, thereby obtaining the classification result of the input image. That is to say, in a conventional deep learning recognition model, the fully connected layer of the deep learning recognition model is used to calculate the logistic regression scores of image features and key prompt words represented by specific labels. The actual function of this fully connected layer is to determine the similarity between the features of the content described by the specific label and the features of the image, that is, to calculate the similarity between the features of the content described by the specific label and the features of the image by adding and dividing by the vector length of each text multimodal feature and image multimodal feature.

[0179] Therefore, based on the structural characteristics of the conventional deep learning recognition model and the model characteristics of the CLIP model, the model structure of a target image recognition model provided in this specific embodiment is based on the CLIP model, by adjusting the model structure of the CLIP model, converting the CLIP model into a conventional deep learning recognition model, and performing model training on the converted model to achieve model tuning. In this way, while ensuring the excellent performance of the CLIP model itself, such as the ability to complete image recognition without collecting data, the cost of training and tuning the model is greatly reduced.

[0180] However, after obtaining the target image recognition model after structural transformation, since only a small number of samples are used to fine-tune the target image recognition model, the amount is too small compared to the amount of data used when training the pre-training data under normal circumstances. Therefore, fine-tuning the target image recognition model will cause the feature distribution learned by the target image recognition model on the sample data to shift, thereby causing the generalization effect of the fine-tuned target image recognition model on other business data sets to deteriorate when fine-tuning is not performed. For example, there are business data sets A and business data sets B for the same object but different business scenarios, where the fine-tuning data set is collected on business data set A. When the above target image recognition model is fine-tuned using business data set A, the reasoning performance of the fine-tuned target image recognition model on business data set B will be reduced, that is, the recognition accuracy of the fine-tuned target image recognition model for images collected by business data set B will be reduced.

[0181] The above specific embodiment is introduced below in combination with specific implementation steps, wherein the sample object in this specific embodiment is a tattoo, and the sample object is the target object in the embodiment of the present application.

[0182] Step 1: Set the initial key prompt words, and after combining the key prompt words to obtain each sample prompt word, use the text encoder to convert each sample prompt word into a corresponding text multimodal feature (also called text multimodal feature, which is the vector feature of the prompt word in the embodiment of the present application).

[0183] Specifically, before model training, multiple initial prompt words for tattoo classification can be set. For example, if the image has a tattoo, the initial prompt word is set to "a photo of person with a tattoo"; if the image does not have a tattoo, the initial prompt word is set to "a photo of person without a tattoo". Based on the combination of the above initial key prompt words, each sample prompt word for model training is obtained. Then, each sample prompt word is input into the text encoder of the CLIP model respectively to obtain the text feature of each sample prompt word, and then each text feature is converted into a text multimodal feature through a text multimodal network. Here, each sample prompt word corresponds to a feature matrix of a text multimodal feature of size [1, 512].

[0184] Among them, the initial key prompt word is the target keyword in the embodiment of the present application; the images included in the business data set A are all images in the first business scenario and containing the target object.

[0185] The specific implementation method corresponding to the above step 1 is similar to the method of obtaining the vector features of the prompt word in the embodiment of the present application, and will not be repeated here.

[0186] Step 2: Convert the obtained text multimodal features into the weights of a fully connected layer, and splice the fully connected layer behind the image encoder of the initial recognition model (also referred to as an image multimodal model) to obtain the recognition model to be optimized; wherein the above-mentioned recognition model to be optimized is the image recognition model to be trained in the embodiment of the present application; the weights of the fully connected layer obtained by the above-mentioned conversion are the initial weight parameters of the fully connected layer in the embodiment of the present application.

[0187] Specifically, considering that the actual role of the fully connected layer in the deep learning recognition model is to calculate the similarity between the features of the content described by the specific label and the features of the image by adding the dot product and dividing by the vector length of the multimodal features of the content described by each specific label and the multimodal features of the image. In the actual reasoning process of the image recognition model, it is also necessary to calculate the similarity between the text multimodal features and the image multimodal features in the above manner. Therefore, it can be considered that the role of the acquired text multimodal features is consistent with that of the fully connected layer in the conventional deep learning recognition model, both of which provide text multimodal features of the prompt words. Therefore, the predetermined text multimodal features in the above step one are converted into fully connected layer weights, and the fully connected layer with the weights added is spliced ​​behind the image encoder to obtain the recognition model to be tuned.

[0188] Since the image encoder sets a fixed preset size for the size of the input image, the image is scaled to the fixed preset size before being input into the image encoder, wherein the fixed preset size is usually 224×224 or 336×336, which are both reasonable and are not specifically limited in the embodiments of the present application.

[0189] In this way, after receiving the input image, the image encoder in the recognition model to be tuned can extract the image features of the image, and pass the image features through the image multimodal network in the image encoder to convert the image features into image multimodal features (vector features of the image in the embodiment of the present application). Here, the image multimodal features corresponding to each image are feature matrices of size [1, 512].

[0190] The specific implementation method corresponding to the above step 2 is similar to the method of obtaining the vector features of the sample image in the embodiment of the present application, and will not be repeated here.

[0191] Step 3: Use the back propagation algorithm to optimize the recognition model to be optimized after splicing, and obtain an image recognition model optimized based on the contrastive learning model.

[0192] Specifically, multiple text multimodal features and image multimodal features are sequentially dot-multiplied and added, and then divided by the vector length of each text multimodal feature and image multimodal feature to calculate the similarity of each text multimodal feature and image multimodal feature, thereby obtaining the feature score between the text feature of each combined prompt word and the feature of the input image. The feature score is used to characterize the similarity between each text feature and the image feature of the image.

[0193] Then, define the feature score as z and perform softmax operation on all feature scores:

[0194]

[0195] σ(z i ) represents the confidence corresponding to the i-th sample prompt word about tattoos, K represents the total number of sample prompt words about tattoos, z i Characterize the feature score corresponding to the i-th sample prompt word, z j Characterize the feature score corresponding to the j-th sample prompt word.

[0196] Through the above formula, the feature score between the text feature of each sample prompt word and the image feature of the image can be mapped to a confidence level between 0 and 1 as the confidence level of the prediction.

[0197] Then, based on the cross-entropy loss function, the loss value between the confidence predicted by the model and the label of the image itself is calculated, and the back propagation algorithm is used to update the weights of the fully connected layer based on the loss value, that is, the text modal feature weights of multiple sample prompt words, and the weight features of the image encoder; wherein, the weights of the above-mentioned fully connected layer, and the text modal feature weights of multiple sample prompt words, are all initial weight parameters of the fully connected layer in the embodiment of the present application, wherein, if this update is not the first update, the updated weights of the fully connected layer are the current weight parameters of the fully connected layer itself in the embodiment of the present application.

[0198] In this way, the above steps 2 and 3 are repeated until all images in the data set are extracted, and gradient descent calculation is performed to optimize the recognition model to be tuned, thereby obtaining the target image recognition model provided in the embodiment of the present application.

[0199] The above steps 1 to 3 can be regarded as one round of model training. Repeat multiple rounds of training until the preset number of training rounds is reached, or the loss value reaches a preset value, then stop the training.

[0200] Compared with the use of traditional models and traditional training methods, this embodiment only requires a small number of samples to obtain a higher model accuracy, greatly reducing the difficulty of data collection in scenarios with long-tail distribution data characteristics such as audit business. Moreover, compared with the training process of the existing comparative learning model, in this embodiment, only the image encoder and the fully connected layer of the recognition model to be tuned need to be trained, that is, by training one image recognition model, the effect of training two encoders can be achieved. In other words, by reducing the number of encoders that need to be trained during model training, the number of training samples and training time required during training are reduced, thereby improving the training efficiency of the image recognition model.

[0201] Moreover, in the subsequent reasoning process of the trained image recognition model, the text encoder can be discarded, and only the weight parameters of the image encoder and the fully connected layer can be retained, thereby improving the image recognition efficiency of the subsequent image recognition model.

[0202] Then, after the tuned model (the target image recognition model in the embodiment of the present application) is obtained through training through the above steps one to three, the tuned model can be tuned again through the following step four.

[0203] Step 4: Select different thresholds and perform weighted averaging of the tuned model and the original model to obtain an adjusted target image recognition model with better generalization performance; wherein the original model is the pre-trained comparative learning model in the embodiment of the present application.

[0204] Specifically, the above step 4 may include:

[0205] Step 1: Select different weight thresholds between 0 and 1, for example, select weights 0, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 and 1 at intervals of 0.1. The weight thresholds and weights are the weights in the weight parameter group of the embodiment of the present application.

[0206] Step 2: Select any weight from step 1, and perform weighted averaging of the model parameters of the model obtained by tuning through steps 1 to 3 above and the original model before tuning according to the selected weight.

[0207] For example, if a weight of 0.1 is selected, the weight parameter of the tuned model is set to 0.1, the weight parameter of the tuned model is set to 0.9, and the values ​​of each position parameter of the original model before tuning and the tuned model are weighted averaged to obtain a new model after weighted averaging; wherein the new model after weighted averaging is the candidate image recognition model in the embodiment of the present application; the values ​​of each position parameter are the weight parameters of the target fully connected layer, the first encoder parameters, the vector feature group and the second encoder parameters in the embodiment of the present application.

[0208] Step 3: Test the performance index of the new model after weighted averaging on the business data set B. The object targeted by the training data contained in the business data set B is the target object in the embodiment of the present application, but the image in the training data belongs to the second business scenario; the above performance index is the evaluation value used to characterize the recognition accuracy of the model in the embodiment of the present application.

[0209] The specific implementation method of the above step 3 is the specific implementation method of step S204 in the embodiment of the present application, which will not be repeated here.

[0210] Step 4: Repeat the above steps 2 and 3 until the model parameters of the tuned model are weighted averaged using all the selected weights to obtain multiple new models; among the multiple new models, select the model with the highest performance index on the business data set B. It can be considered that the selected model can perform image recognition on business data set A (the object targeted by the training data contained in the data set A is the target object in the embodiment of the present application, but the image in the training data belongs to the first business scenario) and can also perform image recognition on business data set B, thereby ensuring that the model has the best generalization performance on business data set B.

[0211] Corresponding to the above method embodiment, the present application also provides a model adjustment device, such as Figure 7 FIG. 1 is a schematic diagram of a structure of a model adjustment device provided in an embodiment of the present application, wherein the device comprises:

[0212] The parameter determination module 701 is used to determine the first model parameter to be adjusted of the target image recognition model and the second model parameter of the pre-trained contrastive learning model; wherein the target image recognition model is an image recognition model for identifying the target object obtained by training the contrastive learning model based on the sample image in the first business scenario, and the first model parameter and the second model parameter are parameters having the same function in the image recognition process;

[0213] The parameter processing module 702 is used to obtain a plurality of weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain a target model parameter corresponding to the weight parameter group; wherein each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter; and the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value;

[0214] A parameter updating module 703 is used to replace the first model parameter of the target image recognition model with the target model parameter for each target model parameter, so as to obtain each candidate image recognition model;

[0215] An evaluation value determination module 704 is used to input a test image into each candidate image recognition model, obtain an output result corresponding to the test image, and determine an evaluation value corresponding to the candidate image recognition model for characterizing the recognition accuracy of the model based on the output result corresponding to the test image and a label of the test image; wherein the test image is an image in a second business scenario and is provided with a label, and the label is used to characterize the presentation status of the target object;

[0216] The model determination module 705 is used to select the candidate image recognition model with the largest evaluation value among the candidate image recognition models as the target image recognition model matching the second business scenario.

[0217] From the above, it can be seen that the solution provided by the embodiment of the present application can, when adjusting the model, fuse the model parameters of the target image recognition model and the model parameters of the corresponding comparative learning model according to the weight parameter group, so as to provide the target image recognition model with the data features required for image recognition of the same object in the second business scenario through parameter fusion, thereby improving the recognition accuracy of the image recognition model for the same object in any business scenario, so as to improve the application generalization of the image recognition model.

[0218] Furthermore, the adjusted target image recognition model is a candidate image recognition model with the largest evaluation value determined based on the output results of the model and the label of the test image. That is to say, the adjusted target image recognition model is the recognition model with the highest recognition accuracy among all candidate recognition models when performing image recognition on images containing target objects in the second business scenario. Therefore, applying a model adjustment method provided in an embodiment of the present application can improve the accuracy of the adjusted target image recognition model when performing recognition on images containing target objects in the second business scenario while ensuring the recognition accuracy of the adjusted target image recognition model for images containing target objects in the first business scenario.

[0219] Optionally, in one implementation, the target image recognition model includes a fully connected layer and an image encoder in the contrastive learning model;

[0220] The parameter determination module 701 is specifically used for:

[0221] Determine a weight parameter of a fully connected layer of the target image recognition model and a first encoder parameter of an image encoder of the target image recognition model as a first model parameter of the target image recognition model;

[0222] Determining a vector feature group generated based on a pre-trained contrastive learning model and a second encoder parameter of an image encoder of the contrastive learning model as a second model parameter of the contrastive learning model;

[0223] The vector feature group is obtained by concatenating the vector features of each sample prompt word generated by the text encoder in the contrastive learning model.

[0224] Optionally, in one implementation, the target model parameters include target weight parameters and target encoder parameters;

[0225] The parameter processing module 702 includes:

[0226] For each weight parameter group, taking the first weight in the group as the weighted weight of the weight parameter of the fully connected layer and the second weight in the group as the weighted weight of the vector feature group, performing weighted summation on the weight parameter and the vector feature group to obtain a target weight parameter;

[0227] The first weight in the group is used as the weighted weight of the first encoder parameter and the second weight in the group is used as the weighted weight of the second encoder parameter, and the first encoder parameter and the second encoder parameter are weightedly summed to obtain the target encoder parameter.

[0228] Optionally, in one implementation, the output result corresponding to the test image includes a similarity between a vector feature of a sample prompt word and a vector feature of the test image;

[0229] The evaluation value determination module 704 includes:

[0230] A confidence determination submodule, used to determine the confidence corresponding to each sample prompt word based on the similarity between the vector feature of each sample prompt word and the vector feature of the test image; wherein the confidence corresponding to any sample prompt word is used to characterize the presentation state described in the sample prompt word, which is the predicted probability of the presentation state of the target object in the test image;

[0231] The evaluation value determination submodule is used to determine the evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model based on the confidence corresponding to each sample prompt word and the label of the test image.

[0232] Optionally, in an implementation manner, the confidence determination submodule is specifically used to:

[0233] The similarity between the vector feature of each sample prompt word and the vector feature of the test image is normalized to obtain the confidence corresponding to the sample prompt word.

[0234] Optionally, in one implementation, the training process of the target image recognition model includes:

[0235] Acquire a sample image; wherein the sample image is provided with a label for characterizing a presentation state of a target object;

[0236] Input the sample image into the image recognition model to be trained to obtain an output result corresponding to each sample prompt word; wherein the image recognition model includes a fully connected layer with a predetermined initial weight parameter and an image encoder in the contrastive learning model; the output result corresponding to each sample prompt word is used to characterize the similarity between the vector feature of the sample prompt word and the vector feature of the sample image;

[0237] Determine a loss value corresponding to the image recognition model based on the output result corresponding to each sample prompt word and the label of the sample image;

[0238] When it is determined based on the loss value that the image recognition model does not meet the model convergence condition, based on the loss value, the current weight parameters of the fully connected layer and the image encoder are adjusted, and the process returns to the step of obtaining the sample image;

[0239] When it is determined based on the loss value that the image recognition model meets the model convergence condition, the model training is terminated to obtain the target image recognition model.

[0240] Optionally, in one implementation, the step of inputting the sample image into an image recognition model to be trained to obtain an output result corresponding to each sample prompt word includes:

[0241] The sample image is input into the image encoder in the image recognition model to be trained so that the image encoder generates vector features of the sample image, and the vector features of the sample image are transmitted to the fully connected layer so as to process the vector features of the sample image based on the current weight parameters of the fully connected layer itself to obtain the output result corresponding to each sample prompt word.

[0242] Optionally, in one implementation, a method for determining the initial weight parameters of the fully connected layer includes:

[0243] The vector feature of each sample prompt word is obtained, and the vector feature of each sample prompt word is concatenated as a row of matrix elements to obtain the initial weight parameter of the fully connected layer.

[0244] The present application also provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0245] Memory 803, used for storing computer programs;

[0246] The processor 801 is used to implement any model adjustment method provided in the above-mentioned embodiments of the present application when executing the program stored in the memory 803.

[0247] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0248] The communication interface is used for communication between the above terminal and other devices.

[0249] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0250] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0251] In another embodiment provided in the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the model adjustment method described in any one of the above embodiments is implemented.

[0252] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the model adjustment method described in any one of the above embodiments.

[0253] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0254] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0255] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, electronic device embodiment, computer-readable storage medium embodiment, and computer program product embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0256] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A model adjustment method, characterized in that: The method comprises: Determine a first model parameter to be adjusted of a target image recognition model and a second model parameter of a pre-trained contrastive learning model; wherein the target image recognition model is an image recognition model for identifying a target object obtained by training the contrastive learning model based on sample images in a first business scenario; the first model parameter and the second model parameter are parameters having the same function in an image recognition process; Acquire multiple weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain a target model parameter corresponding to the weight parameter group; wherein each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter; and the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value; For each target model parameter, using the target model parameter to replace the first model parameter of the target image recognition model, to obtain each candidate image recognition model; For each candidate image recognition model, a test image is input into the candidate image recognition model to obtain an output result corresponding to the test image, and based on the output result corresponding to the test image and a label of the test image, an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model is determined; wherein the test image is an image in a second business scenario and is provided with a label, and the label is used to characterize the presentation state of the target object; A candidate image recognition model with the largest evaluation value among the candidate image recognition models is selected as a target image recognition model matching the second business scenario.

2. The method according to claim 1, characterized in that The target image recognition model includes a fully connected layer and an image encoder in the contrastive learning model; The step of determining the first model parameter of the target image recognition model to be adjusted and the second model parameter of the pre-trained contrastive learning model includes: Determine a weight parameter of a fully connected layer of the target image recognition model and a first encoder parameter of an image encoder of the target image recognition model as a first model parameter of the target image recognition model; Determining a vector feature group generated based on a pre-trained contrastive learning model and a second encoder parameter of an image encoder of the contrastive learning model as a second model parameter of the contrastive learning model; The vector feature group is obtained by concatenating the vector features of each sample prompt word generated by the text encoder in the contrastive learning model.

3. The method according to claim 2, characterized in that The target model parameters include target weight parameters and target encoder parameters; The step of performing weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group for each weight parameter group to obtain the target model parameter corresponding to the weight parameter group includes: For each weight parameter group, taking the first weight in the group as the weighted weight of the weight parameter of the fully connected layer and the second weight in the group as the weighted weight of the vector feature group, performing weighted summation on the weight parameter and the vector feature group to obtain a target weight parameter; The first weight in the group is used as the weighted weight of the first encoder parameter and the second weight in the group is used as the weighted weight of the second encoder parameter, and the first encoder parameter and the second encoder parameter are weightedly summed to obtain the target encoder parameter.

4. The method according to claim 1, characterized in that: The output result corresponding to the test image includes the similarity between the vector feature of a sample prompt word and the vector feature of the test image; The step of determining, based on the output result corresponding to the test image and the label of the test image, an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model comprises: Based on the similarity between the vector feature of each sample prompt word and the vector feature of the test image, the confidence corresponding to the sample prompt word is determined; wherein the confidence corresponding to any sample prompt word is used to characterize the presentation state described in the sample prompt word, which is the predicted probability of the presentation state of the target object in the test image; Based on the confidence level corresponding to each sample prompt word and the label of the test image, an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model is determined.

5. The method according to claim 4, characterized in that The determining of the confidence level corresponding to each sample prompt word based on the similarity between the vector feature of each sample prompt word and the vector feature of the test image includes: The similarity between the vector feature of each sample prompt word and the vector feature of the test image is normalized to obtain the confidence corresponding to the sample prompt word.

6. The method according to claim 2, characterized in that The training process of the target image recognition model includes: Acquire a sample image; wherein the sample image is provided with a label for characterizing a presentation state of a target object; Input the sample image into the image recognition model to be trained to obtain an output result corresponding to each sample prompt word; wherein the image recognition model includes a fully connected layer with a predetermined initial weight parameter and an image encoder in the contrastive learning model; the output result corresponding to each sample prompt word is used to characterize the similarity between the vector feature of the sample prompt word and the vector feature of the sample image; Determine a loss value corresponding to the image recognition model based on the output result corresponding to each sample prompt word and the label of the sample image; When it is determined based on the loss value that the image recognition model does not meet the model convergence condition, based on the loss value, the current weight parameters of the fully connected layer and the image encoder are adjusted, and the process returns to the step of obtaining the sample image; When it is determined based on the loss value that the image recognition model meets the model convergence condition, the model training is terminated to obtain the target image recognition model.

7. The method according to claim 6, characterized in that The step of inputting the sample image into the image recognition model to be trained to obtain an output result corresponding to each sample prompt word includes: The sample image is input into the image encoder in the image recognition model to be trained so that the image encoder generates vector features of the sample image, and the vector features of the sample image are transmitted to the fully connected layer so as to process the vector features of the sample image based on the current weight parameters of the fully connected layer itself to obtain the output result corresponding to each sample prompt word.

8. The method according to claim 6, characterized in that The method for determining the initial weight parameters of the fully connected layer includes: The vector feature of each sample prompt word is obtained, and the vector feature of each sample prompt word is concatenated as a row of matrix elements to obtain the initial weight parameter of the fully connected layer.

9. A model adjustment device, characterized in that: The device comprises: A parameter determination module, used to determine a first model parameter to be adjusted for a target image recognition model, and a second model parameter of a pre-trained contrastive learning model; wherein the target image recognition model is an image recognition model for identifying a target object obtained by training the contrastive learning model based on sample images in a first business scenario; the first model parameter and the second model parameter are parameters having the same function in an image recognition process; a parameter processing module, configured to obtain a plurality of weight parameter groups, and for each weight parameter group, perform weighted sum processing on the first model parameter and the second model parameter according to the weight parameter group to obtain a target model parameter corresponding to the weight parameter group; wherein each weight parameter group includes a first weight assigned to the first model parameter and a second weight assigned to the second model parameter; and the sum of the first weight and the second weight in each weight parameter group is a predetermined sum value; A parameter updating module, for replacing the first model parameter of the target image recognition model with the target model parameter for each target model parameter, to obtain each candidate image recognition model; An evaluation value determination module is used to input a test image into each candidate image recognition model, obtain an output result corresponding to the test image, and determine an evaluation value corresponding to the candidate image recognition model and used to characterize the recognition accuracy of the model based on the output result corresponding to the test image and a label of the test image; wherein the test image is an image in a second business scenario and is provided with a label, and the label is used to characterize the presentation state of the target object; The model determination module is used to select the candidate image recognition model with the largest evaluation value among the candidate image recognition models as the target image recognition model matching the second business scenario.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 8 when executing a program stored in a memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

Images