An image detection method and device, electronic equipment and storage medium

Abstract

The application provides an image detection method and device, electronic equipment and storage medium, comprising: collecting an image to be processed; using a pre-trained image encoder to extract features of the image to be processed to obtain target image features; inputting training set text features and the target image features into a pre-trained contrast network to perform feature matching calculation to obtain a target image query result value; performing similarity calculation on the target image features and training set image features to generate a target similarity matrix; performing nonlinear mapping calculation on labels and the target similarity matrix based on first target hyperparameters to generate a target similarity query result value; performing weighted calculation on the target image query result value and the target similarity query result value based on second target hyperparameters to obtain a target classification result value; and the target classification result value is used to indicate a detection result of the image to be processed. Thus, the application can be more conveniently and quickly applied to various different application environments.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an image detection method, apparatus, electronic device and storage medium. Background Technology

[0002] Surveillance and access control systems often require image detection of massive amounts of collected images to filter out abnormal images and perform corresponding processing operations. Existing technologies can use language-image comparison networks for abnormal image detection.

[0003] Among them, the Language-Image Contrast Network is a multimodal model based on contrastive learning. Its training data is text-image pairs, that is, an image and its corresponding text description. Through contrastive learning, the Language-Image Contrast Network can learn the matching relationship of text-image pairs.

[0004] However, the training process of language-image contrast networks requires a large amount of training data, while in real-world applications, only a small amount of data can be obtained for some occasional anomalies. Therefore, current image detection methods based on language-image contrast networks are not very adaptable to different anomalies. Summary of the Invention

[0005] To address the aforementioned technical problems, this application discloses an image detection method, apparatus, electronic device, and storage medium.

[0006] In a first aspect, this application discloses an image detection method, the method comprising:

[0007] Acquire the image to be processed;

[0008] The image to be processed is used to extract features to obtain target image features;

[0009] The training set text features and the target image features are input into a pre-trained contrast network to perform feature matching calculations and obtain the target image query result value; the training set text features are obtained by pre-trained text encoders extracting features from the training text, and the training text corresponds to the labels of the training images;

[0010] The similarity between the target image features and the training set image features is calculated to generate a target similarity matrix; the training set image features are obtained by the pre-trained image encoder extracting features from the training images.

[0011] Based on the first target hyperparameter, the label and the target similarity matrix are nonlinearly mapped and calculated to generate the target similarity query result value;

[0012] Based on the second target hyperparameter, the target image query result value and the target similarity query result value are weighted and calculated to obtain the target classification result value; the target classification result value is used to indicate the detection result of the image to be processed.

[0013] Optionally, the step of using a pre-trained image encoder to extract features from the image to be processed to obtain target image features includes:

[0014] The image to be processed is used to extract features to obtain a target image feature sequence;

[0015] The target image feature sequence is regularized to form target image features.

[0016] Optionally, the step of performing a nonlinear mapping calculation between the label and the target similarity matrix based on the first hyperparameter to generate a target similarity query result value includes:

[0017] The tag is encoded using a single-hot encoding method and converted into single-hot data.

[0018] The unique hot data and the target similarity matrix are nonlinearly mapped to generate target similarity query result values.

[0019] Optionally, before generating the target similarity query result value by performing a non-linear mapping calculation between the label and the target similarity matrix based on the first target hyperparameter, the process includes:

[0020] Obtain the initial training set image features, validation set image features, and training set text features;

[0021] The text features of the training set and the image features of the validation set are input into the pre-trained comparison network to perform feature matching calculations and obtain the query result value of the validation image.

[0022] The similarity between the features of the training set images and the features of the verification images is calculated to generate a verification similarity matrix;

[0023] Based on the first initial hyperparameters, the label and the verification similarity matrix are nonlinearly mapped to generate verification similarity query result values.

[0024] Based on the second initial hyperparameter, the verification image query result value and the verification similarity query result value are weighted and calculated to obtain the verification classification result value; the verification classification result value is used to indicate the detection result of the verification image;

[0025] Based on the verification classification result value and the corresponding label of the verification image, determine the accuracy of the first initial hyperparameter and the second initial hyperparameter;

[0026] Adjust the values ​​of the first initial hyperparameter and the second initial hyperparameter according to the preset step size, and return to the step of obtaining the initial training set image features, validation set image features and training set text features until the values ​​of the first initial hyperparameter and the second initial hyperparameter exceed the preset hyperparameter search range.

[0027] The first initial hyperparameter and the second initial hyperparameter with the highest accuracy are respectively used as the first target hyperparameter and the second target hyperparameter.

[0028] Optionally, obtaining the initial training set image features, validation set image features, and training set text features includes:

[0029] Acquire training images and verification images; both the training images and the verification images have corresponding labels, and each label corresponds to a different training text.

[0030] Using the pre-trained image encoder, feature extraction is performed on the training image and the validation image respectively to obtain the initial training set image features and the validation set image features;

[0031] The pre-trained text encoder is used to extract features from the training text to obtain the training set text features.

[0032] Optionally, obtaining the training image and the verification image includes:

[0033] Obtain a preset image dataset and randomly divide the preset image dataset into training images and validation images.

[0034] Secondly, this application discloses an image detection apparatus, the apparatus comprising:

[0035] The acquisition module is used to acquire images to be processed.

[0036] The image feature extraction module is used to extract features from the image to be processed using a pre-trained image encoder to obtain target image features;

[0037] The comparison module is used to input the text features of the training set and the features of the target image into the pre-trained comparison network, perform feature matching calculation, and obtain the target image query result value; the text features of the training set are obtained by the pre-trained text encoder to extract features from the training text, and the labels of the training text and the training image correspond;

[0038] The similarity calculation module is used to calculate the similarity between the target image features and the training set image features to generate a target similarity matrix; the training set image features are obtained by the pre-trained image encoder extracting features from the training images;

[0039] The mapping module is used to perform nonlinear mapping calculations between the label and the target similarity matrix based on the first target hyperparameters to generate target similarity query result values;

[0040] The weighting module is used to perform weighted calculations on the target image query result value and the target similarity query result value based on the second target hyperparameter to obtain the target classification result value; the target classification result value is used to indicate the detection result of the image to be processed.

[0041] Thirdly, this application discloses an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the image detection method as described above.

[0042] Fourthly, this application discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image detection method as described in any of the preceding claims.

[0043] Fifthly, this application discloses a computer program product comprising a computer program that, when executed by a processor, implements the image detection method according to any of the preceding claims.

[0044] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:

[0045] The process involves: acquiring the image to be processed; extracting features from the image using a pre-trained image encoder to obtain target image features; inputting the training set text features and target image features into a pre-trained contrastive network for feature matching calculation to obtain the target image query result value; the training set text features are obtained by the pre-trained text encoder from the training text, and the training text corresponds to the labels of the training images; calculating the similarity between the target image features and the training set image features to generate a target similarity matrix; the training set image features are obtained by the pre-trained image encoder from the training images; based on the first target hyperparameter, performing a non-linear mapping calculation between the labels and the target similarity matrix to generate the target similarity query result value; and based on the second target hyperparameter, performing a weighted calculation on the target image query result value and the target similarity query result value to obtain the target classification result value; the target classification result value is used to indicate the detection result of the image to be processed.

[0046] In this way, based on the pre-trained image encoder, pre-trained text encoder, and pre-trained contrastive network, and combined with the first target hyperparameter and the second target hyperparameter, image detection of the image to be processed in any scene can be achieved without adjusting the model parameters of the pre-trained model. The optimal first target hyperparameter and the second target hyperparameter can be found by searching a small amount of anomalous data, which makes it more convenient and faster to apply to various application environments. Attached Figure Description

[0047] Figure 1 This is a flowchart of the steps of an image detection method according to this application;

[0048] Figure 2 This is a schematic diagram of an image detection method according to this application;

[0049] Figure 3 This is a structural block diagram of an image detection device according to this application;

[0050] Figure 4 This is a block diagram of an image detection electronic device according to this application. Detailed Implementation

[0051] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Reference Figure 1 The diagram illustrates a flowchart of an image detection method according to this application, which may specifically include the following steps:

[0053] In step S11, the image to be processed is acquired.

[0054] In some scenarios, it is necessary to perform image detection on the acquired images to filter out abnormal images and perform corresponding processing operations. For example, in a surveillance system, it is necessary to detect images of vehicles violating regulations, or in an access control system, it is necessary to detect images of unfamiliar persons, and so on.

[0055] In related technologies, abnormal image detection can be performed based on language image contrast networks. However, the training process of language image contrast networks requires a large amount of training data. In practical application scenarios, only a small amount of data can be obtained for some occasional abnormal situations. Therefore, there is an urgent need for an image detection method that requires fewer abnormal images during the training process and has strong adaptability.

[0056] In this application, firstly, an image to be processed can be acquired. The image to be processed is the image data that needs to be detected. It can be any image format, a single image or multiple images, and there is no specific limitation.

[0057] In step S12, a pre-trained image encoder is used to extract features from the image to be processed, thereby obtaining the target image features.

[0058] The pre-trained image encoder is part of a pre-trained language image contrast network. In this application, the same pre-trained image encoder can be used for various different scenarios. In this way, the training of the language image contrast network does not require the collection of a large number of abnormal images for different scenarios, thereby improving the adaptability of the language image contrast network.

[0059] In this step, a pre-trained image encoder is used to extract features from the image to be processed to obtain target image features. This includes: using the pre-trained image encoder to extract features from the image to be processed to obtain a target image feature sequence; and performing regularization processing on the target image feature sequence to form target image features.

[0060] In this way, the obtained target image features are more standardized, which facilitates subsequent image detection steps based on the target image features.

[0061] In step S13, the training set text features and target image features are input into the pre-trained contrast network to perform feature matching calculation and obtain the target image query result value; the training set text features are obtained by the pre-trained text encoder to extract features from the training text, and the labels of the training text and the training image correspond.

[0062] In this application, training images have corresponding labels. These labels can indicate the category to which the training image belongs, or they can indicate the content displayed by the training image; the specific meaning is not limited. Different labels correspond to different training texts; in other words, the training texts are used to describe the labels. Therefore, it can be understood that the training texts and training images also have a corresponding degree of matching.

[0063] In this step, the pre-trained contrast network is part of a pre-trained language-image contrast network. The same pre-trained contrast network can be used for various different scenarios. In this way, the training of the language-image contrast network does not require the collection of a large number of abnormal images for different scenarios, thereby improving the adaptability of the language-image contrast network.

[0064] In step S14, the similarity between the target image features and the training set image features is calculated to generate a target similarity matrix; the training set image features are obtained by pre-trained image encoders extracting features from the training images.

[0065] The pre-trained image encoder is part of a pre-trained language image contrast network. In this application, the same pre-trained image encoder can be used for various different scenarios. In this way, the training of the language image contrast network does not require the collection of a large number of abnormal images for different scenarios, thereby improving the adaptability of the language image contrast network.

[0066] In this application, the cosine similarity algorithm, or algorithms such as Jaccard similarity coefficient or Pearson correlation coefficient, can be used to determine the target similarity matrix, and no specific limitation is made.

[0067] In step S15, based on the first target hyperparameter, the label and the target similarity matrix are nonlinearly mapped and calculated to generate the target similarity query result value.

[0068] In one implementation, based on a first hyperparameter, a nonlinear mapping calculation is performed between the label and the target similarity matrix to generate a target similarity query result value. This includes: performing one-hot encoding on the label to convert it into one-hot data; and performing a nonlinear mapping calculation between the one-hot data and the target similarity matrix to generate a target similarity query result value.

[0069] In other words, after converting the labels into one-hot data, nonlinear mapping calculations are performed. It can be understood that the structure of one-hot data is relatively simple, which is beneficial to improving the efficiency of image detection.

[0070] Specifically, the following formula can be used to perform a non-linear mapping calculation on the label-target similarity matrix:

[0071]

[0072] Where s represents the query result value, A represents the first target hyperparameter, C represents the one-hot encoding based on tag text features, and M represents the similarity matrix.

[0073] Alternatively, other algorithms can be used to perform nonlinear mapping calculations on the label-target similarity matrix, and this application does not limit this.

[0074] In step S16, based on the second target hyperparameter, the target image query result value and the target similarity query result value are weighted and calculated to obtain the target classification result value; the target classification result value is used to indicate the detection result of the image to be processed.

[0075] The second target hyperparameter can correspond to the weight of the target image query result value, or it can correspond to the weight of the target similarity query result value, or it can be used as a parameter to calculate the weight of the target image query result value and / or the target similarity query result value. No specific limitation is made.

[0076] By setting a second target hyperparameter, the accuracy of target classification results for different scenarios can be further improved, thereby increasing image detection efficiency.

[0077] In this application, before generating the target similarity query result value by performing a nonlinear mapping calculation between the label and the target similarity matrix based on the first target hyperparameter, it is necessary to first determine the values ​​of the first target hyperparameter and the second target hyperparameter. The values ​​of the first target hyperparameter and the second target hyperparameter are different for different application scenarios. This allows for the implementation of corresponding image detection functions for different scenarios, thereby improving the efficiency of image detection.

[0078] Specifically, determining the values ​​of the first objective hyperparameter and the second objective hyperparameter includes the following steps:

[0079] Obtain the initial training set image features, validation set image features, and training set text features;

[0080] The text features from the training set and the image features from the validation set are input into a pre-trained contrast network to perform feature matching calculations and obtain the query result value of the validation image.

[0081] The similarity between the features of the training set images and the features of the validation images is calculated to generate a validation similarity matrix;

[0082] Based on the first initial hyperparameters, the labels and the verification similarity matrix are non-linearly mapped to generate verification similarity query result values.

[0083] Based on the second initial hyperparameter, the verification image query result value and the verification similarity query result value are weighted and calculated to obtain the verification classification result value; the verification classification result value is used to indicate the detection result of the verification image;

[0084] Based on the verification classification result value and the corresponding label of the verification image, determine the accuracy of the first initial hyperparameter and the second initial hyperparameter;

[0085] Adjust the values ​​of the first initial hyperparameter and the second initial hyperparameter according to the preset step size, and return to the steps of obtaining the initial training set image features, validation set image features and training set text features until the values ​​of the first initial hyperparameter and the second initial hyperparameter exceed the preset hyperparameter search range.

[0086] The first and second initial hyperparameters with the highest accuracy are used as the first target hyperparameter and the second target hyperparameter, respectively.

[0087] In other words, after obtaining the initial training set image features, validation set image features, and training set text features through the hyperparameter search scheme, the values ​​of the first and second initial hyperparameters are continuously adjusted according to a set step size until the hyperparameter search range is completed. Then, based on the recorded accuracy under different hyperparameter conditions, the set of hyperparameters with the highest accuracy is selected as the final hyperparameters for subsequent inference calculations.

[0088] The preset step size can be set to different values ​​depending on the application scenario, and there is no specific limitation.

[0089] In one implementation, obtaining the initial training set image features, the validation set image features, and the training set text features includes:

[0090] Acquire training and validation images; both training and validation images have corresponding labels, each label corresponding to a different training text; use a pre-trained image encoder to extract features from the training and validation images respectively to obtain the initial training set image features and validation set image features; use a pre-trained text encoder to extract features from the training text to obtain the training set text features.

[0091] As can be seen from the foregoing, the pre-trained image encoder, pre-trained text encoder, and pre-trained contrast network in this application are all part of a pre-trained language image contrast network. The same pre-trained image encoder can be used for various different scenarios. In this way, the training of the language image contrast network does not require the collection of a large number of abnormal images for different scenarios, thereby improving the adaptability of the language image contrast network.

[0092] The process of obtaining training and validation images includes: obtaining a preset image dataset and randomly dividing the preset image dataset into training and validation images.

[0093] This further reduces the amount of data required during training, requiring only multiple random samplings of a pre-defined image dataset, thereby further improving the adaptability of the image detection method.

[0094] like Figure 2 The diagram shown is a schematic representation of an image detection method according to this application. It includes two parts: hyperparameter search and inference calculation.

[0095] First, during the hyperparameter search process, the dataset is divided into training and validation data according to a fixed ratio, with the specific data being randomly selected. A one-hot encoder converts the training data's category labels into one-hot data, i.e., one-hot encoding based on label text features. Using the image encoder of the pre-trained language image contrast network model, the training image data is converted into image feature sequences, and the sequence data is regularized to form the training set image features. Using the text encoder of the pre-trained language image contrast network model, the classification label description text data corresponding to the training images is converted into training set text features. The validation set image data is used as inference data. The image encoder of the pre-trained language image contrast network model converts the validation image data into image feature sequences, and the sequence data is regularized to form image feature query data. Based on the language image contrast network... The process involves: matching text features from the training set with image features from the query data to obtain image query results; calculating the similarity between image features from the training set and the query data to generate a similarity matrix; performing a non-linear mapping calculation between the one-hot encoding based on label text features and the similarity matrix, and designing a first initial hyperparameter in the calculation formula to generate similarity query results; performing a weighted calculation on the image query results and the similarity query results, and designing a second initial hyperparameter in the weighted calculation to obtain the final classification result; adjusting the values ​​of the first and second initial hyperparameters according to a set step size; adjusting the data composition of the training and validation sets; performing the next round of calculations until the hyperparameter search range is completed; and selecting the hyperparameter set with the highest accuracy under different recorded hyperparameter conditions as the final hyperparameters for subsequent inference calculations.

[0096] Then, during the inference calculation process, the image data collected in actual work is used as the inference data input. The image encoder of the pre-trained model of the language-image contrast network is used to convert the verification image data into an image feature sequence, and the sequence data is regularized to form image feature query data. Based on the definition of the language-image contrast network, the text features of the training set and the image feature query data are matched to obtain the image query result value. The similarity between the image features of the training set and the image feature query data is calculated to generate a similarity matrix. The one-hot encoding based on the label text features in the hyperparameter search process is nonlinearly mapped to the similarity matrix to generate the similarity query result value. The image query result value and the similarity query result value are weighted to obtain the final classification result value of this inference.

[0097] As can be seen from the above, the technical solution provided by the embodiments of this disclosure, based on a pre-trained image encoder, a pre-trained text encoder, and a pre-trained contrast network, combined with a first target hyperparameter and a second target hyperparameter, can achieve image detection of images to be processed in any scene without adjusting the model parameters of the pre-trained model. The optimal first target hyperparameter and second target hyperparameter can be found by searching a small amount of hyperparameters based on a small amount of abnormal data, thus making it more convenient and faster to apply in various application environments.

[0098] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by this application.

[0099] Reference Figure 3 The diagram shows a structural block diagram of an image detection device according to this application. The device may specifically include the following modules:

[0100] Acquisition module 201 is used to acquire images to be processed;

[0101] The image feature extraction module 202 is used to extract features from the image to be processed using a pre-trained image encoder to obtain target image features;

[0102] The comparison module 203 is used to input the training set text features and the target image features into the pre-trained comparison network, perform feature matching calculation, and obtain the target image query result value; the training set text features are obtained by pre-trained text encoders extracting features from the training text, and the training text corresponds to the label of the training image;

[0103] The similarity calculation module 204 is used to calculate the similarity between the target image features and the training set image features to generate a target similarity matrix; the training set image features are obtained by the pre-trained image encoder extracting features from the training images;

[0104] The mapping module 205 is used to perform nonlinear mapping calculation between the label and the target similarity matrix based on the first target hyperparameter, and generate target similarity query result value;

[0105] The weighting module 206 is used to perform weighted calculation on the target image query result value and the target similarity query result value based on the second target hyperparameter to obtain the target classification result value; the target classification result value is used to indicate the detection result of the image to be processed.

[0106] As can be seen from the above, the technical solution provided by the embodiments of this disclosure, based on a pre-trained image encoder, a pre-trained text encoder, and a pre-trained contrast network, combined with a first target hyperparameter and a second target hyperparameter, can achieve image detection of images to be processed in any scene without adjusting the model parameters of the pre-trained model. The optimal first target hyperparameter and second target hyperparameter can be found by searching a small amount of hyperparameters based on a small amount of abnormal data, thus making it more convenient and faster to apply in various application environments.

[0107] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0108] Figure 4 This is a block diagram illustrating an image detection electronic device according to an exemplary embodiment.

[0109] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of an electronic device to perform the above-described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0110] In an exemplary embodiment, a computer program product is also provided, which, when run on a computer, enables the computer to implement the above-described image detection method.

[0111] As can be seen from the above, the technical solution provided by the embodiments of this disclosure, based on a pre-trained image encoder, a pre-trained text encoder, and a pre-trained contrast network, combined with a first target hyperparameter and a second target hyperparameter, can achieve image detection of images to be processed in any scene without adjusting the model parameters of the pre-trained model. The optimal first target hyperparameter and second target hyperparameter can be found by searching a small amount of hyperparameters based on a small amount of abnormal data, thus making it more convenient and faster to apply in various application environments.

[0112] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0113] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This application is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0118] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0119] The above provides a detailed description of the image detection method, apparatus, electronic device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image detection method, characterized in that, include: Acquire the image to be processed; The image to be processed is used to extract features to obtain target image features; The text features of the training set and the features of the target image are input into a pre-trained contrastive network to perform feature matching calculations and obtain the target image query result value; the text features of the training set are obtained by pre-trained text encoders extracting features from the training text, and the training text corresponds to the label of the training image; The similarity between the target image features and the training set image features is calculated to generate a target similarity matrix; the training set image features are obtained by the pre-trained image encoder extracting features from the training images; Based on the first target hyperparameter, the label and the target similarity matrix are nonlinearly mapped and calculated to generate the target similarity query result value; Based on the second target hyperparameter, the target image query result value and the target similarity query result value are weighted and calculated to obtain the target classification result value; the target classification result value is used to indicate the detection result of the image to be processed; The first target hyperparameter and the second target hyperparameter are obtained in the following way: Obtain the initial training set image features, validation set image features, and training set text features; The text features of the training set and the image features of the validation set are input into the pre-trained comparison network to perform feature matching calculations and obtain the query result value of the validation image. The similarity between the initial training set image features and the verification image features is calculated to generate a verification similarity matrix; Based on the first initial hyperparameters, the labels of the training images are nonlinearly mapped to the verification similarity matrix to generate verification similarity query result values. Based on the second initial hyperparameter, the verification image query result value and the verification similarity query result value are weighted and calculated to obtain the verification classification result value; the verification classification result value is used to indicate the detection result of the verification image; Based on the verification classification result value and the corresponding label of the verification image, determine the accuracy of the first initial hyperparameter and the second initial hyperparameter; Adjust the values ​​of the first initial hyperparameter and the second initial hyperparameter according to the preset step size, and return to the step of obtaining the initial training set image features, validation set image features and training set text features until the values ​​of the first initial hyperparameter and the second initial hyperparameter exceed the preset hyperparameter search range. The first initial hyperparameter and the second initial hyperparameter with the highest accuracy are respectively used as the first target hyperparameter and the second target hyperparameter.

2. The method according to claim 1, characterized in that, The step of using a pre-trained image encoder to extract features from the image to be processed to obtain target image features includes: The image to be processed is used to extract features to obtain a target image feature sequence; The target image feature sequence is regularized to form target image features.

3. The method according to claim 1, characterized in that, The step of performing a nonlinear mapping calculation between the label and the target similarity matrix based on the first target hyperparameter to generate a target similarity query result value includes: The tag is encoded using a single-hot encoding method and converted into single-hot data. The unique hot data and the target similarity matrix are nonlinearly mapped to generate target similarity query result values.

4. The method according to claim 1, characterized in that, The acquisition of initial training set image features, validation set image features, and training set text features includes: Acquire training images and verification images; both the training images and the verification images have corresponding labels, and each label corresponds to a different training text. Using the pre-trained image encoder, feature extraction is performed on the training image and the validation image respectively to obtain the initial training set image features and the validation set image features; The pre-trained text encoder is used to extract features from the training text to obtain the training set text features.

5. The method according to claim 4, characterized in that, The acquisition of training images and verification images includes: Obtain a preset image dataset and randomly divide the preset image dataset into training images and validation images.

6. An image detection device, characterized in that, include: The acquisition module is used to acquire images to be processed. The image feature extraction module is used to extract features from the image to be processed using a pre-trained image encoder to obtain target image features; The comparison module is used to input the text features of the training set and the features of the target image into the pre-trained comparison network, perform feature matching calculation, and obtain the target image query result value; the text features of the training set are obtained by the pre-trained text encoder to extract features from the training text, and the labels of the training text and the training image correspond; The similarity calculation module is used to calculate the similarity between the target image features and the training set image features to generate a target similarity matrix; the training set image features are obtained by the pre-trained image encoder extracting features from the training images; The mapping module is used to perform nonlinear mapping calculations between the label and the target similarity matrix based on the first target hyperparameters to generate target similarity query result values; The weighting module is used to perform a weighted calculation on the target image query result value and the target similarity query result value based on the second target hyperparameter to obtain a target classification result value; the target classification result value is used to indicate the detection result of the image to be processed; The first target hyperparameter and the second target hyperparameter are obtained in the following way: Obtain the initial training set image features, validation set image features, and training set text features; The text features of the training set and the image features of the validation set are input into the pre-trained comparison network to perform feature matching calculations and obtain the query result value of the validation image. The similarity between the initial training set image features and the verification image features is calculated to generate a verification similarity matrix; Based on the first initial hyperparameters, the labels of the training images are nonlinearly mapped to the verification similarity matrix to generate verification similarity query result values. Based on the second initial hyperparameter, the verification image query result value and the verification similarity query result value are weighted and calculated to obtain the verification classification result value; the verification classification result value is used to indicate the detection result of the verification image; Based on the verification classification result value and the corresponding label of the verification image, determine the accuracy of the first initial hyperparameter and the second initial hyperparameter; Adjust the values ​​of the first initial hyperparameter and the second initial hyperparameter according to the preset step size, and return to the step of obtaining the initial training set image features, validation set image features and training set text features until the values ​​of the first initial hyperparameter and the second initial hyperparameter exceed the preset hyperparameter search range. The first initial hyperparameter and the second initial hyperparameter with the highest accuracy are respectively used as the first target hyperparameter and the second target hyperparameter.

7. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the image detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image detection method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image detection method according to any one of claims 1 to 5.

Images