Image detection methods and related model training methods, devices, equipment, and media

Abstract

The present application discloses an image detection method and a training method, apparatus, device, and medium for a related model, wherein the image detection method includes: image features of multiple images and category correlation of at least one set of image pairs, wherein the multiple images include a reference image and a target image, and each two images in the multiple images form an image pair, and the category correlation indicates the likelihood that the image pairs belong to the same image category; using the category correlation, updating the image features of the multiple images; using the updated image features, obtaining an image category detection result of the target image, and the image category detection result is used to indicate the image category to which the target image belongs. The above scheme can improve the accuracy of image category detection.

Description

Technical Field

[0001] This application is a divisional application of the patent application entitled "Image detection method and training method, device, equipment, and medium for related models" filed by the applicant on October 27, 2020, with application number 2020111674022. This application relates to the field of image processing technology, and in particular to an image detection method and training method, device, equipment, and medium for related models. Background Art

[0002] In recent years, with the advancement of information technology, image category detection has been widely used in many scenarios, including facial recognition and video surveillance. For example, in facial recognition, image category detection can be used to classify multiple facial images, helping to distinguish a user-specified face from a group of images. Generally speaking, the accuracy of image category detection is often one of the primary performance indicators. Therefore, improving the accuracy of image category detection has become a highly valuable research topic. Summary of the Invention

[0003] The present application provides an image detection method and a training method, apparatus, equipment, and medium for related models.

[0004] In a first aspect, the present application provides an image detection method, comprising: obtaining image features of a plurality of images and category correlation of at least one group of image pairs, wherein the plurality of images include a reference image and a target image, and every two images in the plurality of images constitute a group of image pairs, and the category correlation indicates the possibility that the image pairs belong to the same image category; using the category correlation, updating the image features of the plurality of images; using the updated image features, obtaining an image category detection result of the target image; wherein the image category detection result is used to indicate the image category to which the target image belongs.

[0005] Therefore, image features of multiple images and the category correlation of at least one set of image pairs are obtained, and the multiple images include a reference image and a target image, and every two images in the multiple images form an image pair. The category correlation indicates the possibility that the image pair belongs to the same image category, and the category correlation is used to update the image features, thereby using the updated image features to obtain the image category detection result of the target image, and the image category detection result is used to indicate the image category to which the target image belongs. Therefore, by using the category correlation to update the image features, the image features corresponding to images of the same image category can be made closer, and the image features corresponding to images of different image categories can be made more distant, which can help improve the robustness of the image features, and help capture the distribution of the image features, and thus can help improve the accuracy of image category detection.

[0006] Among them, using the updated image features to determine the image category detection result of the target image includes: using the updated image features to perform prediction processing to obtain probability information, wherein the probability information includes a first probability value that the target image belongs to at least one reference category, and the reference category is the image category to which the reference image belongs; based on the first probability value, obtaining the image category detection result.

[0007] Therefore, by using the updated image features for prediction processing, probability information is obtained, and the probability information includes a first probability value that the target image belongs to at least one reference category, so that based on the first probability value, the image category detection result is obtained, and then prediction can be performed on the basis of the image features updated by using the category correlation, and the first probability value that the target image belongs to at least one image category is obtained, which can be beneficial to the prediction accuracy.

[0008] The probability information also includes a second probability value that the reference image belongs to at least one reference category; before obtaining the image category detection result based on the first probability value, the method also includes: when the number of times the prediction processing is performed meets the preset conditions, using the probability information to update the category correlation; and re-executing the step of using the category correlation to update the image features of multiple images, and when the number of times the prediction processing is performed does not meet the preset conditions, obtaining the image category detection result based on the first probability value.

[0009] Therefore, by setting the probability information to also include a second probability value that the reference image belongs to at least one reference category, and before obtaining the image category detection result based on the first probability value, further using the probability information to update the category relevance when the number of times the prediction process is performed meets the preset condition, and re-performing the step of updating the image features using the category relevance, and obtaining the image category detection result based on the first probability value when the number of times the prediction process is performed does not meet the preset condition. Therefore, when the number of times the prediction process is performed meets the preset condition, the category relevance can be updated using the first probability value that the target image belongs to at least one reference category and the second probability value that the reference image belongs to at least one reference category, thereby improving the robustness of the category similarity, and continuing to use the updated category similarity to update the image features, thereby further improving the robustness of the image features, thereby enabling the category similarity and the image features to promote and complement each other, and when the number of times the prediction process is performed does not meet the preset condition, the image category detection result is obtained based on the first probability value, which can be conducive to further improving the accuracy of image category detection.

[0010] Among them, the category relevance includes: the final probability value of each group of image pairs belonging to the same image category; using the probability information to update the category relevance, including: taking each image in the multiple images as the current image, and taking the image pair containing the current image as the current image pair; obtaining the sum of the final probability values ​​of all current image pairs of the current image as the probability sum of the current image; and using the first probability value and the second probability value to respectively obtain the reference probability value that each group of current image pairs belongs to the same image category; using the probability sum and the reference probability value respectively to adjust the final probability value of each group of current image pairs.

[0011] Therefore, the category relevance is set to include the final probability value of each group of image pairs belonging to the same image category, and each image in the multiple images is used as the current image, and the image pair containing the current image is used as the current image pair, so as to obtain the final probability value of all current image pairs of the current image as the probability sum of the current image, and use the first probability value and the second probability value to respectively obtain the reference probability value of each group of image pairs belonging to the same image category, and then use the probability sum and the reference probability value to adjust the final probability value of each group of current image pairs. Therefore, the reference probability value of each group of current image pairs belonging to the same image category can be used to update the category relevance, which can be beneficial to the image category to which the aggregated images belong and improve the accuracy of the category relevance.

[0012] Among them, the updated image features are used for prediction processing to obtain probability information, including: using the updated image features to predict the predicted categories to which the target image and the reference image belong, wherein the predicted category belongs to at least one reference category; for each group of image pairs, the category comparison results and feature similarities of the image pairs are obtained, and a first matching degree between the category comparison results and the feature similarities of the image pairs is obtained, wherein the category comparison results indicate whether the predicted categories to which the image pairs belong are the same, and the feature similarity indicates the similarity between the image features of the image pairs; and based on the predicted category and the reference category to which the reference image belongs, a second matching degree of the reference image with respect to the predicted category and the reference category is obtained; and the probability information is obtained using the first matching degree and the second matching degree.

[0013] Therefore, using the updated image features, the predicted categories to which the target image and the reference image belong are predicted, and the predicted categories belong to at least one reference category. Thus, for each image pair, a category comparison result and feature similarity are obtained for the image pair, and a first degree of matching between the category comparison result and feature similarity for the image pair is obtained. The category comparison result indicates whether the predicted categories to which the image pair belongs are the same, and the feature similarity indicates the similarity between the image features of the image pair. Based on the predicted category and the reference category to which the reference image belongs, a second degree of matching between the predicted category and the reference category for the reference image is obtained. Probability information is then obtained using the first and second degrees of matching. Therefore, by obtaining the first degree of matching between the category comparison result and similarity for the image pair, the accuracy of image category detection can be characterized from the dimension of any image pair based on the degree of matching between the category comparison result and feature similarity for the predicted category. Furthermore, by obtaining the second degree of matching between the predicted category and the reference category for the reference image, the accuracy of image category detection can be characterized from the dimension of a single image based on the degree of matching between the predicted category and the reference category. Combining the two dimensions of any two images and a single image to obtain probability information can help improve the accuracy of probability information prediction.

[0014] Among them, when the category comparison result is that the predicted categories are the same, the feature similarity is positively correlated with the first matching degree; when the category comparison result is that the predicted categories are different, the feature similarity is negatively correlated with the first matching degree, and the second matching degree when the predicted category is the same as the reference category is greater than the second matching degree when the predicted category is different from the reference category.

[0015] Therefore, when the category comparison result is that the predicted category is the same, the feature similarity is set to be positively correlated with the first matching degree, and when the category comparison result is that the predicted category is different, the feature similarity is set to be negatively correlated with the first matching degree. Therefore, when the category comparison result is that the predicted category is the same, the higher the feature similarity, the higher the first matching degree with the category comparison result, that is, the more the feature similarity matches the category comparison result. When the category comparison result is that the predicted category is different, the higher the feature similarity, the lower the first matching degree with the category comparison result, that is, the less the feature similarity matches the category comparison result. This is beneficial for capturing the possibility that the image category is the same between any two images in the subsequent probability information prediction process, thereby improving the accuracy of the probability information prediction. In addition, since the second matching degree when the predicted category is the same as the reference category is greater than the second matching degree when the predicted category is different from the reference category, it is beneficial for capturing the accuracy of the image features of a single image in the subsequent probability information prediction process, thereby improving the accuracy of the probability information prediction.

[0016] The method of using the updated image features to predict the prediction category to which the image belongs includes: using the updated image features based on a conditional random field network to predict the prediction category to which the image belongs.

[0017] Therefore, by using the updated image features based on the conditional random field network to predict the prediction category to which the target image and the reference image belong, it is beneficial to improve the accuracy and efficiency of the prediction.

[0018] The method of obtaining the probability information by using the first matching degree and the second matching degree includes obtaining the probability information by using the first matching degree and the second matching degree based on cyclic belief propagation.

[0019] Therefore, based on cyclic belief propagation, the first matching degree and the second matching degree are used to obtain probability information, which can help improve the accuracy of the probability information.

[0020] The preset conditions include: the number of times the prediction process is executed does not reach a preset threshold.

[0021] Therefore, setting the preset condition to: the number of times the prediction processing is performed does not reach the preset threshold can help to fully capture the category relationship between images through the preset threshold number of loop iterations during the image category detection process, thereby helping to improve the accuracy of image category detection.

[0022] Among them, the step of updating the image features of multiple images using category relevance is performed by a graph neural network.

[0023] Therefore, by using graph neural networks to perform the above-mentioned steps of using category relevance to update image features, the efficiency of image feature updating can be improved.

[0024] The image features of multiple images are updated by utilizing the category relevance, including: utilizing the category relevance and the image features to obtain intra-category image features and inter-category image features; and utilizing the intra-category image features and the inter-category image features to perform feature conversion to obtain updated image features.

[0025] Therefore, by utilizing category correlation and image features, intra-class image features and inter-class image features are obtained, and feature conversion is performed on the two dimensions of intra-class image features and inter-class image features to obtain updated image features, which can improve the accuracy of image feature updating.

[0026] Among them, the image detection method also includes: when the image pair belongs to the same image category, determining the initial category correlation of the image pair as a preset upper limit value; when the image pair belongs to different image categories, determining the initial category correlation of the image pair as a preset lower limit value; when at least one of the image pairs is a target image, determining the initial category correlation of the image pair as a preset value between the preset lower limit value and the preset upper limit value.

[0027] Therefore, by determining the initial category correlation of the image pair as a preset upper limit value when the image pair belongs to the same image category, and determining the initial category correlation of the image pair as a preset lower limit value when the image pair belongs to different image categories, and determining the initial category correlation of the image pair as a preset numerical value between the preset lower limit value and the preset upper limit value when at least one of the image pairs is a target image, the above-mentioned preset upper limit value, preset lower limit value and preset numerical value can be used to characterize the possibility that the image categories of the image pair are the same, so as to facilitate subsequent processing, thereby improving the convenience and accuracy of characterizing the category correlation.

[0028] A second aspect of the present application provides a training method for an image category detection model, comprising: obtaining sample image features of multiple sample images and sample category correlation of at least one group of sample image pairs, wherein the multiple sample images include a sample reference image and a sample target image, and every two sample images in the multiple sample images form a group of sample image pairs, and the sample category correlation represents the possibility that the sample image pairs belong to the same image category; based on a first network of the image detection model, using the sample category correlation, updating the sample image features of the multiple sample images; based on a second network of the image detection model, using the updated sample image features, obtaining an image category detection result of the sample target image; wherein the image category detection result is used to indicate the image category to which the sample target image belongs; and using the image category detection result of the sample target image and the image category annotated by the sample target image to adjust the network parameters of the image detection model.

[0029] Therefore, sample image features of multiple sample images and sample category correlation of at least one set of sample image pairs are obtained, and the multiple sample images include a sample reference image and a sample target image, and every two sample images in the multiple sample images form a set of sample image pairs, and the sample category correlation represents the possibility that the sample image pairs belong to the same image category, and based on the first network of the image detection model, the sample image features of the multiple sample images are updated using the sample category correlation, so that based on the second network of the image detection model, the image category detection results of the sample target images are obtained using the updated sample image features, and the image category detection results are used to indicate the image category to which the sample target images belong, and then the network parameters of the image detection model are adjusted using the image category detection results and the image category annotated by the sample target images. Therefore, by updating the sample image features using the sample category correlation, the sample image features corresponding to images of the same image category can be made closer, and the sample image features corresponding to images of different image categories can be made more distant, which can be beneficial to improving the robustness of the sample image features, and can be beneficial to capturing the distribution of the sample image features, and can be beneficial to improving the accuracy of the image detection model.

[0030] Among them, based on the second network of the image detection model, using the updated sample image features, an image category detection result of the sample target image is obtained, including: based on the second network, using the updated sample image features to perform prediction processing to obtain sample probability information, wherein the sample probability information includes a first sample probability value that the sample target image belongs to at least one reference category and a second sample probability value that the sample reference image belongs to at least one reference category, and the reference category is the image category to which the sample reference image belongs; based on the first sample probability value, an image category detection result of the sample target image is obtained; before using the image category detection result of the sample target image and the image category annotated by the sample target image to adjust the network parameters of the image detection model, the method also includes: using the first sample probability value and the second sample probability value to update the sample category correlation; using the image category detection result of the sample target image and the image category annotated by the sample target image, adjusting the network parameters of the image detection model, including: using the first sample probability value and the image category annotated by the sample target image to obtain a first loss value of the image detection model; and using the actual category correlation between the sample target image and the sample reference image and the updated sample category correlation to obtain a second loss value of the image detection model; adjusting the network parameters of the image detection model based on the first loss value and the second loss value.

[0031] Therefore, based on the second network, the updated sample image features are used for prediction processing to obtain sample probability information, and the sample probability information includes a first sample probability value that the sample target image belongs to at least one reference category and a second sample probability value that the sample reference image belongs to at least one reference category, and the reference category is the image category to which the sample reference image belongs, so that based on the first sample probability value, the image category detection result of the sample target image is obtained, and the first sample probability value and the second sample probability value are used to update the sample category correlation, and then the first sample probability value and the image category annotated by the sample target image are used to obtain the first loss value of the image detection model, and the actual category correlation and the updated sample category correlation between the sample target image and the sample reference image are used to obtain the second loss value of the image detection model, so that based on the first loss value and the second loss value, the network parameters of the image detection model are adjusted. Therefore, the network parameters of the image detection model can be adjusted from the dimension of the category correlation between the two images and the dimension of the image category of a single image, which can help improve the accuracy of the image detection model.

[0032] The image detection model includes at least one sequentially connected network layer, each network layer includes a first network and a second network; before adjusting the network parameters of the image detection model based on the first loss value and the second loss value, the method also includes: when the current network layer is not the last network layer of the image detection model, using the next network layer of the current network layer, re-executing the step of updating the sample image features based on the first network of the image detection model using the sample category correlation and subsequent steps until the current network layer is the last network layer of the image detection model; adjusting the network parameters of the image detection model based on the first loss value and the second loss value, including: using the first weight corresponding to each network layer to weight the first loss value corresponding to each network layer to obtain a first weighted loss value; and using the second weight corresponding to each network layer to weight the second loss value corresponding to each network layer to obtain a second weighted loss value; adjusting the network parameters of the image detection model based on the first weighted loss value and the second weighted loss value; wherein, the later the network layer is in the image detection model, the larger the first weight and the second weight corresponding to the network layer are.

[0033] Therefore, the image detection model is set to include at least one sequentially connected network layer, and each network layer includes a first network and a second network. When the current network layer is not the last network layer of the image detection model, the next network layer of the current network layer is used to re-execute the first network based on the image detection model, and the sample category correlation is used to update the sample image features and subsequent steps until the current network layer is the last network layer of the image detection model, thereby using the first weight corresponding to each network layer to weight the first loss value corresponding to each network layer to obtain a first weighted loss value. , and use the second weight corresponding to each network layer to weight the second loss value corresponding to each network layer to obtain the second weighted loss value, and then adjust the network parameters of the image detection model based on the first weighted loss value and the second weighted loss value. The later the network layer is in the image detection model, the larger the first weight and the second weight corresponding to the network layer are. The loss value corresponding to the network layer of each layer of the image detection model can be obtained, and the weight corresponding to the later network layer is set to be larger, so that the data obtained by processing each network layer can be fully utilized to adjust the network parameters of image detection, which is beneficial to improving the accuracy of the image detection model.

[0034] The third aspect of the present application provides an image detection device, including an image acquisition module, a feature update module and a result acquisition module, the image acquisition module is used to acquire image features of multiple images and category correlation of at least one group of image pairs, and the multiple images include a reference image and a target image, and every two images in the multiple images form a group of image pairs, and the category correlation indicates the possibility that the image pairs belong to the same image category; the feature update module is used to use the category correlation to update the image features of the multiple images; the result acquisition module is used to use the updated image features to obtain the image category detection result of the target image; wherein the image category detection result is used to indicate the image category to which the target image belongs.

[0035] The fourth aspect of the present application provides a training device for an image detection model, comprising a sample acquisition module, a feature update module, a result acquisition module and a parameter adjustment module, wherein the sample acquisition module is used for sample image features of multiple sample images and sample category correlation of at least one group of sample image pairs, and the multiple sample images include sample reference images and sample target images, and every two sample images in the multiple sample images form a group of sample image pairs, and the sample category correlation indicates the possibility that the sample image pairs belong to the same image category; the feature update module is used to update the sample image features of the multiple sample images based on the first network of the image detection model using the sample category correlation; the result acquisition module is used to obtain the image category detection result of the sample target image based on the second network of the image detection model using the updated sample image features; wherein the image category detection result is used to indicate the image category to which the sample target image belongs; the parameter update module is used to adjust the network parameters of the image detection model using the image category detection result of the sample target image and the image category annotated by the sample target image.

[0036] In a fifth aspect, the present application provides an electronic device comprising a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the image detection method in the first aspect above, or to implement the image detection model training method in the second aspect above.

[0037] In a sixth aspect, the present application provides a computer-readable storage medium having program instructions stored thereon. When the program instructions are executed by a processor, the image detection method in the first aspect described above is implemented, or the training method of the image detection model in the second aspect described above is implemented.

[0038] The above scheme obtains image features of multiple images and the category correlation of at least one set of image pairs, and the multiple images include a reference image and a target image, and every two images in the multiple images form an image pair. The category correlation indicates the possibility that the image pair belongs to the same image category, and the category correlation is used to update the image features, thereby using the updated image features to obtain the image category detection result of the target image, and the image category detection result is used to indicate the image category to which the target image belongs. Therefore, by using the category correlation to update the image features, the image features corresponding to images of the same image category can be made closer, and the image features corresponding to images of different image categories can be made more distant, which can help improve the robustness of the image features, and help capture the distribution of the image features, and thus can help improve the accuracy of image category detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of an embodiment of the image detection method of the present application;

[0040] Figure 2 This is a flow chart of another embodiment of the image detection method of the present application;

[0041] Figure 3 This is a flow chart of another embodiment of the image detection method of the present application;

[0042] Figure 4 This is a schematic diagram of the state of an embodiment of the image detection method of the present application;

[0043] Figure 5 This is a flowchart of an embodiment of a training method for an image detection model of the present application;

[0044] Figure 6 This is a flowchart of another embodiment of the training method of the image detection model of the present application;

[0045] Figure 7 This is a schematic diagram of the framework of an embodiment of the image detection device of the present application;

[0046] Figure 8 This is a schematic diagram of a framework of an embodiment of a training device for an image detection model of the present application;

[0047] Figure 9 This is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0048] Figure 10 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0050] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0051] The terms "system" and "network" are often used interchangeably in this document. The term "and / or" is simply a description of an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " generally indicates that the related objects are in an "or" relationship. Furthermore, "multiple" in this document means two or more than two.

[0052] The image detection method of the present application can be used to detect the image category of an image. The image category can be specifically set according to the actual application situation. For example, in order to distinguish whether the image belongs to a "person" or an "animal", the image category can be set to include: people, animals; or, in order to distinguish whether the image belongs to a "male" or a "female", the image category can be set to include: male, female. In addition, it should be noted that the image detection method of the present application can be specifically used for monitoring cameras (or computers, tablets and other electronic devices connected to monitoring cameras), so that after the image is captured, the image detection method disclosed in the present application can be used to detect the image category to which it belongs; or, the image detection method of the present application can also be used for computers, tablets and other electronic devices, so that after the image is acquired, the image detection method disclosed in the present application can be used to detect the image category to which it belongs. Please refer to the following disclosed embodiments of the present application for details.

[0053] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the image detection method of the present application. Specifically, it may include the following steps:

[0054] Step S11: Obtain image features of multiple images and category relevance of at least one set of image pairs.

[0055] In the embodiment of the present disclosure, the multiple images include a target image and a reference image. Specifically, the target image is an image of unknown image category, while the reference image is an image of known image category.

[0056] In one implementation scenario, to improve the efficiency of extracting image features, an image detection model can be pre-trained, and the image detection model includes a feature extraction network for extracting image features of the target image and the reference image. The specific training process can be referred to the steps in the embodiment of the training method of the image detection model of this application, and will not be described in detail here.

[0057] In a specific implementation scenario, the feature extraction network may include a sequentially connected backbone network, a pooling layer, and a fully connected layer. The backbone network may specifically be any one of a convolutional network and a residual network (e.g., ResNet12). The convolutional network may include several (e.g., 4) convolutional blocks, each of which includes a sequentially connected convolutional layer, a batch normalization layer, and an activation layer (e.g., ReLu). In addition, the last several (e.g., the last 2) convolutional blocks in the convolutional network may also include a dropout layer. The pooling layer may specifically be a global average pooling (GAP) layer.

[0058] In a specific implementation scenario, after the target image and the reference image are processed by the above-mentioned feature extraction network, image features of a preset dimension (eg, 128 dimensions) can be obtained. Specifically, the image features can be represented in the form of vectors.

[0059] In the disclosed embodiments, every two images in the plurality of images constitute an image pair. For example, if the plurality of images includes reference image A, reference image B, and target image C, then the image pairs may include: reference image A and target image C, reference image B and target image C, and reference image A and target image C. Other scenarios can be deduced in this manner and are not further exemplified here.

[0060] In one implementation scenario, the category relevance of the likelihood that an image pair belongs to the same image category may specifically include: a final probability value of the image pair belonging to the same image category. For example, when the final probability value is 0.9, it can be considered that the image pair has a high probability of belonging to the same image category; or, when the final probability value is 0.1, it can be considered that the image pair has a low probability of belonging to the same image category; or, when the final probability value is 0.5, it can be considered that the image pair has an equal probability of belonging to the same image category and a probability of belonging to different image categories.

[0061] In a specific implementation scenario, when the steps in the embodiments of the present disclosure are started, the category correlation of the image pairs belonging to the same image category can be initialized. Specifically, when the image pairs belong to the same image category, the initial category correlation of the image pairs can be determined as a preset upper limit value. For example, when the category correlation is represented by the above-mentioned final probability value, the preset upper limit value can be set to 1. In addition, when the image pairs belong to different image categories, the initial category correlation of the image pairs is determined as a preset lower limit value. For example, when the category correlation is represented by the above-mentioned final probability value, the preset lower limit value can be set to 0. In addition, since the target image is the image to be detected, when at least one of the image pairs is the target image, the category correlation of the image pairs belonging to the same image category cannot be determined. In order to improve the robustness of the initialized category correlation, the category correlation can be determined as a preset value between the preset lower limit value and the preset upper limit value. For example, when the category correlation is represented by the above-mentioned final probability value, the preset value can be set to 0.5. Of course, it can also be set to 0.4, 0.6, or 0.7 as needed, which is not limited here.

[0062] In another specific implementation scenario, for the convenience of description, when the category correlation is represented by the final probability value, the final probability value initialized between the i-th image and the j-th image in the target image and the reference image can be recorded as In addition, there are N types of reference images, and each image category corresponds to K reference images. When the 1st to NKth images are reference images, the image categories marked by the i-th reference image and the j-th reference image can be recorded as y i ,y j , then the final probability value of the image pair belonging to the same image category is recorded as It can be expressed as:

[0063]

[0064] Therefore, when there are T target images, that is, when the NK+1th to NK+Tth images are target images, the category correlation of the image pairs can be expressed as a (NK+T)*(NK+T) matrix.

[0065] In one implementation scenario, image categories can be specifically set based on the actual application scenario. For example, in a face recognition scenario, image categories can be based on age, specifically including "children," "adolescents," "elderly," etc.; or, in a medical image classification scenario, image categories can be based on angiography duration, specifically including "arterial phase," "portal phase," "delayed phase," etc. Other scenarios can be similarly categorized, and are not listed here.

[0066] In a specific implementation scenario, as mentioned above, there may be a total of N reference images of image categories, and each image category corresponds to K reference images, where N is an integer greater than or equal to 1, and K is an integer greater than or equal to 1. That is, the image detection method embodiment of the present disclosure can be used in scenarios where reference images labeled with image categories are relatively scarce, such as medical image classification detection, rare species image classification detection, etc.

[0067] In one implementation scenario, the number of target images may be one. In other implementation scenarios, the number of target images may be multiple based on actual application needs. For example, in a face recognition scenario in video surveillance, the image data of the facial region detected in each frame of the captured video may be used as the target image. In this case, the number of target images may also be two, three, four, and so on. Other scenarios can be similarly deduced, and are not listed here one by one.

[0068] Step S12: using the category relevance, updating the image features of the multiple images.

[0069] In one implementation scenario, in order to improve the efficiency of updating image features, as mentioned above, an image detection model can be pre-trained, and the image detection model further includes a graph neural network (GNN). The specific training process can refer to the relevant steps in the training method embodiment of the image detection model disclosed in this application, which will not be described here. On this basis, the image features of each image can be used as nodes of the input image data of the graph neural network. For the sake of convenience of description, the initialized image features can be recorded as V0 gnn , and the category relevance of any image pair is used as the edge between nodes. For the convenience of description, the initialized category relevance can be recorded as E0 gnn , so that the graph neural network can be used to perform the steps of updating image features by using category correlation, which can be specifically expressed as:

[0070] V1 gnn =f(V0 gnn ,E0 gnn )……(2)

[0071] In the above formula (1), f() represents the graph neural network, V1 gnn Represents the updated image features.

[0072] In a specific implementation scenario, as described above, when the category relevance of image pairs is represented as a (NK+T)*(NK+T) matrix, the input image data of the graph neural network can be viewed as a directed graph. Furthermore, when the two images contained in any two image pairs are non-repeating, the input image data corresponding to the graph neural network can also be viewed as an undirected graph, without limitation here.

[0073] In one implementation scenario, in order to improve the accuracy of image features, the category correlation and image features can be used to obtain intra-category image features and inter-category image features, wherein the intra-category image features are image features obtained by aggregating image features within a category using category correlation, while the inter-category image features are image features obtained by aggregating image features between categories using category correlation. gnn Represents the initialized image features, E0 gnn The category correlation obtained by initialization, the image feature within the category can be expressed as E0 gnn V0 gnn , the inter-class image feature can be expressed as (1-E0 gnn )V0 gnnAfter obtaining the intra-class image features and inter-class image features, the intra-class image features and inter-class image features can be used to perform feature transformation to obtain updated image features. Specifically, the intra-class image features and inter-class image features can be spliced ​​to obtain fused image features, and the nonlinear transformation function f can be used to transform the image features. θ The fused image features are transformed to obtain updated image features, which can be specifically expressed as:

[0074] V1 gnn =f θ (E0 gnn V0 gnn ||(1-E0 gnn )V0 gnn )……(3)

[0075] In the above formula (3), the nonlinear conversion function f θ The parameter is θ, and || represents the splicing operation.

[0076] Step S13: Using the updated image features, obtain the image category detection result of the target image.

[0077] In one implementation scenario, the image category detection result may be used to indicate the image category to which the target image belongs.

[0078] In one implementation scenario, after obtaining the updated image features, the updated image features can be used for prediction processing to obtain probability information, and the probability information includes a first probability value that the target image belongs to at least one reference category, so that the image category detection result can be obtained based on the first probability value. Specifically, the reference category is the image category to which the reference image belongs. For example, the multiple images include reference image A1, reference image A2, reference image A3, reference image A4 and target image C, the image category to which reference image A1 belongs is "plain scan period", the image category to which reference image A2 belongs is "arterial phase", the image category to which reference image A3 belongs is "portal phase", and the image category to which reference image A4 belongs is "delayed phase", then at least one reference category includes: "plain scan period", "arterial phase", "portal phase", and "delayed phase". Other scenarios can be deduced by analogy, and examples are not given one by one here.

[0079] In a specific implementation scenario, to improve prediction efficiency, as described above, an image detection model can be pre-trained, and the image detection model includes a conditional random field (CRF) network. The specific training process can be referred to the relevant description in the training method embodiment of the image detection model of this application, which is not repeated here. In this case, based on the conditional random field (CRF) network, the updated image features can be used to predict a first probability value that the target image belongs to at least one reference category.

[0080] In another specific implementation scenario, the probability information including the first probability value can be directly used as the image category detection result for the target image for user reference. For example, in a medical image category detection scenario, the first probability values ​​for the target image belonging to the "arterial phase," "portal phase," and "delayed phase" can be used as the image category detection result for the target image. Other scenarios can be similarly deduced, and no further examples are given here.

[0081] In another specific implementation scenario, the image category of the target image can also be determined based on the first probability value that the target image belongs to at least one reference category, and the determined image category can be used as the image category detection result of the target image. Specifically, the reference category corresponding to the highest first probability value can be used as the image category of the target image. For example, in a medical image category detection scenario, if the first probability values ​​predicted that the target image belongs to the "arterial phase," "portal phase," and "delayed phase" are 0.1, 0.8, and 0.1, respectively, then the "portal phase" can be used as the image category of the target image. Other scenarios can be deduced in this way, and no further examples are given here.

[0082] In another implementation scenario, prediction processing is performed using the updated image features to obtain probability information, and the probability information includes a first probability value that the target image belongs to at least one reference category and a second probability value that the reference image belongs to at least one reference category. If the number of prediction processing executions meets a preset condition, the probability information can be used to update the category relevance of the multiple images, and the above-mentioned step S12 and subsequent steps, i.e., the steps of updating the image features using the category relevance and performing prediction processing using the updated image features, are re-executed until the number of prediction processing executions does not meet the preset condition. The above-mentioned method can use the first probability value that the target image belongs to at least one reference category and the second probability value that the reference image belongs to at least one reference category to update the category relevance representing the image pair, thereby improving the robustness of the category similarity, and continue to use the updated category similarity to update the image features, thereby further improving the robustness of the image features, thereby enabling the category similarity and image features to promote and complement each other, which can be beneficial to further improve the accuracy of image category detection.

[0083] In a specific implementation scenario, the preset condition may specifically include: the number of times the prediction process is executed does not reach a preset threshold. The preset threshold is at least 1, for example, 1, 2, 3, etc., which is not limited here.

[0084] In another specific implementation scenario, if the number of prediction processes performed does not meet a preset condition, an image category detection result for the target image can be obtained based on the first probability value. For details, please refer to the aforementioned related description and will not be repeated here. Furthermore, the specific process of updating the category relevance using probability information can be referred to the relevant steps in the following disclosed embodiments and will not be repeated here.

[0085] In one implementation scenario, taking medical image classification as an example, several medical images obtained by scanning the object to be inspected (such as a patient, etc.) are obtained as several target images, and arterial phase medical images, portal phase medical images, and delayed phase medical images are given as reference images. Thus, every two images in the above reference images and target images can be combined into an image pair, and the initial category correlation of the image pairs can be obtained. At the same time, the initial image features of each image are extracted, and then the image features of the above multiple images are updated using the category correlation. The updated image features are used to obtain the image category detection results of the above several target images, for example, the first probability values ​​of the above several target images belonging to the "arterial phase", "portal phase", and "delayed phase" respectively. Other scenarios can be deduced by analogy, and no further examples are given here.

[0086] The above scheme includes image features of multiple images and category correlation of at least one set of image pairs, wherein the multiple images include a reference image and a target image, and each two images in the multiple images form an image pair. The category correlation indicates the likelihood that the image pairs belong to the same image category, and the category correlation is used to update the image features, thereby using the updated image features to obtain the image category detection result of the target image. Therefore, by using the category correlation to update the image features, the image features corresponding to images of the same image category can be made closer, and the image features corresponding to images of different image categories can be made more distant, which can help improve the robustness of the image features and capture the distribution of the image features, thereby helping to improve the accuracy of image category detection.

[0087] See also Figure 2 , Figure 2 This is a flow chart of another embodiment of the image detection method of the present application. Specifically, it may include the following steps:

[0088] Step S21: Obtain image features of multiple images and category relevance of at least one set of image pairs.

[0089] In the disclosed embodiment, the multiple images include a reference image and a target image. Each pair of images in the multiple images constitutes an image pair. The category correlation indicates the likelihood that the image pairs belong to the same image category. For details, please refer to the relevant steps in the previously disclosed embodiment and will not be repeated here.

[0090] Step S22: using the category relevance, updating the image features of the multiple images.

[0091] For details, please refer to the relevant steps in the aforementioned disclosed embodiments, which will not be repeated here.

[0092] Step S23: Perform prediction processing using the updated image features to obtain probability information.

[0093] In the disclosed embodiment, the probability information includes a first probability value that the target image belongs to at least one reference category and a second probability value that the reference image belongs to at least one reference category. The reference category is the image category to which the reference image belongs. For details, please refer to the relevant description in the disclosed embodiment above and will not be repeated here.

[0094] Specifically, the updated image features can be used to predict the predicted categories to which the target image and the reference image belong, and the predicted categories belong to at least one reference category. Taking medical image category detection as an example, when at least one reference category includes "arterial phase," "portal phase," and "delayed phase," the predicted category is any one of "arterial phase," "portal phase," and "delayed phase." Other scenarios can be deduced in this way, and examples are not given here one by one. After obtaining the predicted category, for each group of image pairs, the category comparison results and feature similarity of the image pairs can be obtained, and a first matching degree between the category comparison results and feature similarity of the image pairs is obtained. The category comparison result indicates whether the predicted categories to which the image pairs belong are the same, and the feature similarity indicates the similarity between the image features of the image pairs. Based on the predicted category and the reference category to which the reference image belongs, a second matching degree of the reference image with respect to the predicted category and the reference category is obtained, so that probability information can be obtained using the first matching degree and the second matching degree. The above method, by obtaining the first matching degree of the image pair regarding the category comparison results and similarity, can characterize the accuracy of the image category detection from the dimension of any image pair based on the matching degree between the category comparison results of the predicted category and the feature similarity, and by obtaining the second matching degree of the reference image regarding the predicted category and the reference category, can characterize the accuracy of the image category detection from the dimension of a single image based on the matching degree between the predicted category and the reference category, and combine the two dimensions of any two images and a single image to obtain probability information, which can help improve the accuracy of probability information prediction.

[0095] In one implementation scenario, in order to improve prediction efficiency, the prediction category to which the image belongs can be predicted based on the conditional random field network and using the updated image features.

[0096] In one implementation scenario, when the category comparison result is that the predicted category is the same, the feature similarity is positively correlated with the first matching degree, that is, the greater the feature similarity, the greater the first matching degree, the more the category comparison result matches the feature similarity; conversely, the smaller the feature similarity, the smaller the first matching degree, and the less the category comparison result matches the feature similarity; and when the category comparison result is that the predicted category is different, the feature similarity is negatively correlated with the first matching degree, that is, the greater the feature similarity, the smaller the first matching degree, the less the category comparison result matches the feature similarity; conversely, the smaller the feature similarity, the greater the first matching degree, and the more the category comparison result matches the feature similarity. The above method can help capture the possibility that the image categories between the image pairs are the same in the subsequent probability information prediction process, thereby helping to improve the accuracy of probability information prediction.

[0097] In a specific implementation scenario, for ease of description, a random variable u can be set for the image features of the target image and the reference image. Furthermore, the random variable during the l-th prediction process can be recorded as u l For example, the random variable corresponding to the image features of the i-th image in the 1st to NKth reference images and the NK+1th to NK+Tth target images can be recorded as u i , Similarly, the random variable corresponding to the image feature of the jth image can be recorded as u j The value of the random variable is specifically the predicted category predicted by using the corresponding image features, and the predicted category can be specifically represented by the sequence number of N image categories. Therefore, in the lth prediction process, when the random variable corresponding to the image feature of one of the image pairs The value of (i.e. the corresponding predicted category) is m (i.e. the mth image category), and the random variable corresponding to the image feature of the other is When the value of (i.e., the corresponding predicted category) is n (i.e., the nth image category), the corresponding first matching degree can be recorded as Specifically, it can be expressed as:

[0098]

[0099] In the above formula (4), Indicates the feature similarity between the image features of the i-th image and the image features of the j-th image during the l-th prediction process. Specifically, it can be obtained by the cosine distance. For the convenience of description, the image features of the i-th image during the l-th prediction process can be recorded as And the image features of the jth image during the lth prediction process are recorded as The cosine distance can be used to obtain the feature similarity between the two and normalized to the range of 0 to 1. It can be specifically expressed as:

[0100]

[0101] In the above formula (5), |||| represents the modulus of the image feature.

[0102] In another implementation scenario, when the predicted category is the same as the reference category, the second degree of matching between the reference images is greater than the second degree of matching between the reference images when the predicted category is different from the reference category. This approach facilitates accurately capturing the image features of individual images during subsequent probability information prediction, thereby improving the accuracy of probability information prediction.

[0103] In a specific implementation scenario, as mentioned above, during the lth prediction process, the random variable corresponding to the image feature of the image can be recorded as u l, such as the random variable corresponding to the image feature of the i-th image can be recorded as The value of the random variable is specifically the predicted category predicted by the corresponding image features. As mentioned above, the predicted category can be represented by the sequence number of N image categories. In addition, the image category marked by the i-th image can be recorded as y i Therefore, when the random variable corresponding to the image feature of the reference image When the value of (i.e., the corresponding predicted category) is m (i.e., the mth image category), the corresponding second matching degree can be recorded as Specifically, it can be expressed as:

[0104]

[0105] In the above formula (6), σ represents the tolerance probability when the value of the random variable (i.e., the predicted category) is wrong (i.e., different from the reference category). Specifically, σ can be set to be less than a preset numerical threshold, for example, σ can be set to 0.14, which is not limited here.

[0106] In one implementation scenario, during the first prediction process, a conditional distribution can be obtained based on the first matching degree and the second matching degree, which can be specifically expressed as:

[0107]

[0108] In the above formula (7),<j,k> represents a pair of random variables and And j<k, ∝ indicates positive correlation. From formula (7), we can see that when the first matching degree and the second matching degree are high, the conditional distribution will also be large accordingly. On this basis, for each image, the probability information of the corresponding image can be obtained by summing the conditional distributions corresponding to the random variables corresponding to all images except the image, which can be specifically expressed as:

[0109]

[0110] In the above formula (8), in, represents a random variable The probability value of the image category is the mth reference category. In addition, for the convenience of description, the random variables corresponding to all images in the lth prediction process are represented as V l crf ,in, As mentioned earlier, Represents the random variable corresponding to the image feature of the i-th image during the l-th prediction process.

[0111] In another implementation scenario, in order to improve the accuracy of the probability information, the probability information can be obtained by using the first matching degree and the second matching degree based on Loopy Belief Propagation (LBP). Specifically, for the random variable corresponding to the image feature of the i-th image in the l-th prediction process, Let its probability information be b′ l,i In particular, the probability information can be expressed as b′ l,i is considered as a column vector, and the jth element of the column vector represents the random variable The probability value of taking value j. Therefore, we can give an initial value (b l ,i) 0 And update b′ by iterating t times according to the following rules l,i , until convergence:

[0112]

[0113] In the above formulas (9) and (10), Indicates that it contains random variables to 1*N matrix of information, Indicates the first matching degree, Indicates the second matching degree, N j represents a random variable Other random variables except , / means the division of the corresponding elements of two matrices, Indicates the multiplication of the corresponding elements of the matrix. [] represents the normalization function, that is, it represents the matrix elements within the [] symbol divided by the sum of all elements. In addition, when j>NK, represents the random variable corresponding to the target image. Since the image category of the target image is unknown, its second matching degree is unknown. When the final iteration converges after t′ times, the corresponding probability information b′ l,i =(b l,i ) t′ .

[0114] Step S24: Determine whether the number of times the prediction process is performed meets the preset conditions. If so, execute step S25; otherwise, execute step S27.

[0115] Specifically, the preset condition may include: the number of times the prediction process is executed does not reach a preset threshold value. The preset threshold value is at least 1, for example, 1, 2, 3, etc., which is not limited here.

[0116] Step S25: Using the probability information, update the category relevance.

[0117] In the embodiment of the present disclosure, as mentioned above, the category relevance may specifically include: the final probability value of each image pair belonging to the same image category. For ease of description, the category relevance updated after the first prediction process may be denoted as E l gnn In particular, as mentioned above, before the first prediction process, the initialized category correlation can be recorded as E0 gnn In addition, further, the category relevance E l gnn The final probability value that the i-th image and the j-th image belong to the same image category can be recorded as In particular, the category relevance E0 gnn The final probability value that the i-th image and the j-th image belong to the same image category can be recorded as

[0118] On this basis, each of the multiple images can be used as the current image, and the image pair containing the current image can be used as the current image pair. In the first prediction process, the first probability value and the second probability value can be used to obtain the reference probability value of each group of current image pairs belonging to the same image category. Taking the current image pair containing the i-th image and the j-th image as an example, the reference probability value It can be expressed as:

[0119]

[0120] In the above formula (11), N represents the number of at least one image category. The above formula (11) indicates that for the i-th image and the j-th image, the sum of the products of the probabilities that the random variables corresponding to the two have the same values ​​is obtained.

[0121] At the same time, the sum of the final probability values ​​of all current image pairs of the current image can be obtained as the probability sum of the current image. Specifically, for the l-th prediction process, the updated category relevance can be expressed as E l gnn , the category relevance before updating can be expressed as That is, the category relevance before updating The final probability value that the i-th image and the j-th image belong to the same image category can be recorded as Therefore, for the current image being the i-th image, when the other image in the image pair containing the i-th image is denoted as k, the sum of the final probability values ​​of all current image pairs of the current image can be expressed as

[0122] After obtaining the reference probability value and the probability sum, the final probability value of each image pair can be adjusted using the probability sum and the reference probability value for each current image pair. Specifically, the final probability value of the image pair can be used as a weight, and the weight can be used to perform weighted processing (e.g., weighted averaging) on ​​the reference probability value of the image pair obtained in the last prediction process, and the weighted processing result and the reference probability value can be used to adjust the final probability value. Update to get the final probability value after the update in the lth prediction process Specifically, it can be expressed as:

[0123]

[0124] In the above formula (12), the i-th image represents the current image, and the i-th image and the j-th image form a set of current image pairs. represents the reference probability value of the image pair containing the i-th image obtained by the l-1-th prediction process, represents the reference probability value that the i-th image and the j-th image obtained by the l-th prediction process belong to the same image category, Indicates the final probability value before updating that the i-th image and the j-th image belong to the same image category during the l-th prediction process. Indicates the final probability value after updating that the i-th image and the j-th image belong to the same image category during the l-th prediction process. Represents the sum of the final probability values ​​of all current image pairs for the current image (i.e., the i-th image).

[0125] Step S26: Re-execute step S22.

[0126] After obtaining the updated category relevance, the above step S22 and subsequent steps can be re-executed, that is, the image features of multiple images are updated using the updated category relevance. Specifically, the updated category relevance is denoted as E l gnn , and the image feature V used in the lth prediction process l gnn For example, the above step S22 "updating the image features of multiple images using the category correlation" can be expressed as:

[0127]

[0128] In the above formula (13), It represents the image features used in the l+1th prediction process. For other information, please refer to the relevant description in the aforementioned disclosed embodiment and will not be repeated here.

[0129] Such a cycle can make image features and category relevance promote each other, complement each other, and jointly improve their respective robustness. Therefore, after multiple cycles, a more accurate feature distribution can be captured, which is conducive to improving the accuracy of image category detection.

[0130] Step S27: Obtaining an image category detection result based on the first probability value.

[0131] In one implementation scenario, when the image category detection result includes the image category of the target image, the reference category corresponding to the largest first probability value can be used as the image category of the target image. Specifically, it can be expressed as:

[0132]

[0133] In the above formula (14), represents the image category of the i-th image, represents the first probability value of the i-th image belonging to at least one reference category after L prediction processes, and Y0 represents at least one reference category.

[0134] For details, please refer to the relevant steps in the aforementioned disclosed embodiments, which will not be repeated here.

[0135] Different from the aforementioned embodiment, by setting the probability information to also include a second probability value that the reference image belongs to at least one reference category, and before obtaining the image category detection result based on the first probability value, further using the probability information to update the category relevance when the number of prediction processing executions meets the preset condition, and re-execute the step of updating the image features using the category relevance, and obtaining the image category detection result based on the first probability value when the number of prediction processing executions does not meet the preset condition. Therefore, when the number of prediction processing executions meets the preset condition, the category relevance can be updated using the first probability value that the target image belongs to at least one reference category and the second probability value that the reference image belongs to at least one reference category, thereby improving the robustness of the category similarity, and continuing to use the updated category similarity to update the image features, thereby further improving the robustness of the image features, thereby enabling the category similarity and the image features to promote and complement each other, and when the number of prediction processing executions does not meet the preset condition, the image category detection result is obtained based on the first probability value, which can be conducive to further improving the accuracy of image category detection.

[0136] See also Figure 3 , Figure 3: This is a flow chart of another embodiment of the image detection method of the present application. In the embodiment of the present disclosure, image detection is specifically performed by an image detection model, and the image detection model includes at least one (e.g., L) sequentially connected network layers, each network layer including a first network (e.g., GNN) and a second network (e.g., CRF). The embodiment of the present disclosure may specifically include the following steps:

[0137] Step S31: Obtain image features of multiple images and category relevance of at least one set of image pairs.

[0138] In the disclosed embodiments, the multiple images include a reference image and a target image. Each pair of images in the multiple images constitutes an image pair. The category correlation indicates the likelihood that the image pairs belong to the same image category. For details, please refer to the relevant descriptions in the previously disclosed embodiments and will not be repeated here.

[0139] Please refer to Figure 4 , Figure 4 Schematic diagram of an embodiment of the image detection method of the present application. Figure 4 As shown, the circles in the first network represent image features, the solid squares in the second network represent the image categories annotated in the reference image, and the dashed squares represent target images with unknown image categories. Different fills in the squares and circles correspond to different image categories. Furthermore, the pentagons in the second network represent random variables corresponding to image features.

[0140] In one implementation scenario, the feature extraction network can be considered a network independent of the image detection model. In another implementation scenario, the feature extraction network can also be considered as part of the image detection model. In addition, the specific structure of the feature extraction network can be found in the relevant description of the aforementioned disclosed embodiments and will not be repeated here.

[0141] Step S32: Based on the first network of the lth network layer, the image features of the multiple images are updated using the category correlation.

[0142] Specifically, taking l as 1 as an example, the category correlation obtained by initialization in step S31 can be used to update the image features initialized in step S31 to obtain the following: Figure 4 The image features represented by the circle in the first network layer. When l is other values, it can be combined with Figure 4 And so on, I will not give examples one by one here.

[0143] Step S33: Based on the second network of the lth network layer, the updated image features are used to perform prediction processing to obtain probability information.

[0144] In the embodiment of the present disclosure, the probability information includes a first probability value that the target image belongs to at least one reference category and a second probability value that the reference image belongs to at least one reference category.

[0145] Specifically, taking l as 1 as an example, we can use the image features represented by the circle in the first layer of the network to perform prediction processing and obtain probability information. When l is other values, we can combine Figure 4 And so on, I will not give examples one by one here.

[0146] Step S34: Determine whether the network layer performing the prediction processing is the last network layer of the image detection model. If not, execute step S35; if so, execute step S37.

[0147] Specifically, when the image detection model includes L network layers, it can be determined whether l is less than L. If so, it indicates that there is still a network layer that has not performed the above-mentioned image feature update and probability information prediction steps, then the following step S35 can be continued to use subsequent network layers to continue to update image features and predict probability information. If not, it indicates that all network layers of the image detection model have all performed the above-mentioned image feature update and probability information prediction steps, then the following step S37 can be executed, that is, based on the first probability value in the probability information, to obtain the image category detection result.

[0148] Step S35: Using the probability information, update the category relevance and add 1 to l.

[0149] Specifically, still taking l as 1 as an example, the probability information predicted by the first network layer can be used to update the category relevance, and l+1, that is, l is updated to 2 at this time.

[0150] The specific process of updating the category relevance using the probability information can be found in the relevant description in the aforementioned disclosed embodiments, which will not be repeated here.

[0151] Step S36: Re-execute step S32 and subsequent steps.

[0152] Specifically, still taking l as 1 as an example, after the above step S35, l is updated to 2, and the above step S32 and subsequent steps are re-executed. Figure 4 That is, based on the first network of the second network layer, the image features of multiple images are updated using the category correlation, and based on the second network of the second network layer, the updated image features are used for prediction processing to obtain probability information, and so on. No more examples are given here.

[0153] Step S37: Obtaining an image category detection result based on the first probability value.

[0154] For details, please refer to the relevant description in the aforementioned disclosed embodiments, which will not be repeated here.

[0155] Unlike the previous embodiment, when the prediction process is not performed at the last network layer, probability information is used to update the category relevance, and the next network layer is used again to update the image features of multiple images using the category relevance. Therefore, the robustness of the category similarity can be improved, and the updated category similarity can be used to update the image features, thereby further improving the robustness of the image features. In turn, the category similarity and image features can promote and complement each other, which can further improve the accuracy of image category detection.

[0156] See also Figure 5 , Figure 5 This is a flowchart of an embodiment of the training method of the image detection model of the present application. Specifically, it may include the following steps:

[0157] Step S51: obtaining sample image features of a plurality of sample images and sample category correlations of at least one set of sample image pairs.

[0158] In the disclosed embodiments, the plurality of sample images includes a sample reference image and a sample target image. Each pair of sample images in the plurality of sample images forms a sample image pair, and the sample category correlation indicates the likelihood that the sample image pair belongs to the same image category. The process for obtaining sample image features and sample category correlation can be found in the aforementioned disclosed embodiments for obtaining image features and category correlation, and will not be further elaborated here.

[0159] In addition, the sample target images, sample reference images and image categories can also refer to the relevant descriptions of the target images, reference images and image categories in the aforementioned disclosed embodiments, which will not be repeated here.

[0160] In one implementation scenario, the sample image features can be specifically extracted by a feature extraction network. The feature extraction network can be independent of the image detection model in the disclosed embodiments, or it can be part of the image detection model in the disclosed embodiments, without limitation herein. The specific structure of the feature extraction network can be found in the relevant description in the aforementioned disclosed embodiments and will not be repeated here.

[0161] It should be noted that, unlike the aforementioned disclosed embodiments, during the training process, the image category of the sample target image is known, and specifically, the image category to which the sample target image belongs can be marked on the sample target image.

[0162] Step S52: Based on the first network of the image detection model, sample image features of the plurality of sample images are updated using sample category correlation.

[0163] In one implementation scenario, the first network may be a GNN. Sample category relevance may be used as edges of the GNN input image data, and sample image features may be used as points of the GNN input image data. The GNN may then be used to process the input image data to update the sample image features. For details, please refer to the relevant descriptions in the aforementioned disclosed embodiments, which will not be repeated here.

[0164] Step S53: Based on the second network of the image detection model, the image category detection result of the sample target image is obtained using the updated sample image features.

[0165] In one implementation scenario, the second network may be a conditional random field (CRF) network. Based on the CRF, the updated sample image features may be used to obtain an image category detection result for the sample target image. Specifically, the image category detection result may include a first sample probability value that the sample target image belongs to at least one reference category, where the reference category is the image category to which the sample reference image belongs.

[0166] Step S54: using the image category detection result of the sample target image and the image category annotated by the sample target image, adjust the network parameters of the image detection model.

[0167] Specifically, the cross-entropy loss function can be used to calculate the difference between the image category detection results of the sample target image and the image category annotated on the sample target image, thereby obtaining the loss value of the image detection model and adjusting the network parameters of the image detection model accordingly. Furthermore, if the feature extraction network is independent of the image detection model, the network parameters of both the image detection model and the feature extraction network can be adjusted based on the loss value.

[0168] In an implementation scenario, stochastic gradient descent (SGD), batch gradient descent (BGD), mini-batch gradient descent (MBGD), etc. can be used to adjust the network parameters using the loss value. Batch gradient descent means that all samples are used to update the parameters in each iteration; stochastic gradient descent means that one sample is used to update the parameters in each iteration; mini-batch gradient descent means that a batch of samples is used to update the parameters in each iteration, which will not be repeated here.

[0169] In one implementation scenario, a training end condition may be set. When the training end condition is met, the training may be terminated. Specifically, the training end condition may include any of the following: the loss value is less than a preset loss threshold, the current training number of times reaches a preset number threshold (for example, 500 times, 1000 times, etc.), which are not limited here.

[0170] In another implementation scenario, based on the second network, the updated sample image features can be used for prediction processing to obtain sample probability information, and the sample probability information includes a first sample probability value that the sample target image belongs to at least one reference category and a second sample probability value that the sample reference image belongs to at least one reference category, thereby obtaining an image category detection result of the sample target image based on the first sample probability value, and before adjusting the network parameters of the image detection model using the image category detection result of the sample target image and the image category annotated by the sample target image, using the first sample probability value and the second sample probability value, updating the sample category correlation, thereby obtaining a first loss value of the image detection model using the first sample probability value and the image category annotated by the sample target image, and obtaining a second loss value of the image detection model using the actual category correlation and the updated sample category correlation between the sample target image and the sample reference image, and then adjusting the network parameters of the image detection model based on the first loss value and the second loss value. The above method can adjust the network parameters of the image detection model from the dimension of the category correlation between the two images and the dimension of the image category of a single image, thereby being conducive to improving the accuracy of the image detection model.

[0171] In a specific implementation scenario, the specific process of performing prediction processing based on the second network using the updated sample image features to obtain sample probability information can be found in the description of performing prediction processing using the updated image features to obtain probability information in the aforementioned disclosed embodiments, and will not be repeated here. Furthermore, the specific process of updating the sample category relevance using the first sample probability value and the second sample probability value can be found in the description of updating the category relevance using probability information in the aforementioned disclosed embodiments, and will not be repeated here.

[0172] In another specific implementation scenario, a cross entropy loss function may be used to calculate a first loss value between the first sample probability value and the image category annotated by the sample target image.

[0173] In another specific implementation scenario, a binary cross entropy loss function can be used to calculate a second loss value between the actual category correlation between the sample target image and the sample reference image and the updated sample category correlation. Specifically, when the image categories of the image pairs are the same, the actual category correlation of the corresponding image pairs can be set to a preset upper limit value (e.g., 1); when the image categories of the image pairs are different, the actual category correlation of the corresponding image pairs can be set to a lower limit value (e.g., 0). For ease of description, the actual category correlation can be denoted as c ij .

[0174] In another specific implementation scenario, the weights corresponding to the first loss value and the second loss value can be used to weight the first loss value and the second loss value, respectively, to obtain weighted loss values, and the weighted loss values ​​can be used to adjust the network parameters. Specifically, the weight corresponding to the first loss value can be set to 0.5, and the weight corresponding to the second loss value can also be set to 0.5, to indicate that the first loss value and the second loss value are equally important when adjusting the network parameters. In addition, the corresponding weights can also be adjusted according to the different importance of the first loss value and the second loss value, and examples will not be given one by one here.

[0175] The above scheme obtains sample image features of multiple sample images and sample category correlation of at least one group of sample image pairs, and the multiple sample images include sample reference images and sample target images, and every two sample images in the multiple sample images form a group of sample image pairs. The sample category correlation indicates the possibility that the sample image pairs belong to the same image category, and based on the first network of the image detection model, the sample image features of the multiple sample images are updated using the sample category correlation, so that based on the second network of the image detection model, the image category detection results of the sample target images are obtained using the updated sample image features, and then the network parameters of the image detection model are adjusted using the image category detection results and the image categories annotated by the sample target images. Therefore, by using the sample category correlation to update the sample image features, the sample image features corresponding to images of the same image category can be made closer, and the sample image features corresponding to images of different image categories can be made more distant, which can be beneficial to improving the robustness of the sample image features, and can be beneficial to capturing the distribution of the sample image features, which can be beneficial to improving the accuracy of the image detection model.

[0176] See also Figure 6 , Figure 6 This is a flow chart of another embodiment of the training method for the image detection model of the present application. In the disclosed embodiment, the image detection model includes at least one (e.g., L) sequentially connected network layers, each of which includes a first network and a second network. Specifically, the following steps may be included:

[0177] Step S601: Obtain sample image features of a plurality of sample images and sample category correlations of at least one set of sample image pairs.

[0178] In the disclosed embodiment, the plurality of sample images include a sample reference image and a sample target image. Every two sample images in the plurality of sample images form a set of sample image pairs. The sample category correlation indicates the possibility that the sample image pairs belong to the same image category.

[0179] For details, please refer to the relevant steps in the aforementioned disclosed embodiments, which will not be repeated here.

[0180] Step S602: Based on the first network of the lth network layer, sample image features of the plurality of sample images are updated using sample category correlation.

[0181] For details, please refer to the relevant steps in the aforementioned disclosed embodiments, which will not be repeated here.

[0182] Step S603: Based on the second network of the lth network layer, the updated sample image features are used to perform prediction processing to obtain sample probability information.

[0183] In the embodiment of the present disclosure, the sample probability information includes a first sample probability value that the sample target image belongs to at least one reference category and a second sample probability value that the sample reference image belongs to at least one reference category. The at least one reference category is the image category to which the sample reference image belongs.

[0184] For details, please refer to the relevant steps in the aforementioned disclosed embodiments, which will not be repeated here.

[0185] Step S604: Based on the first sample probability value, obtain the image category detection result of the sample target image corresponding to the lth network layer.

[0186] For the convenience of description, the image category detection result of the i-th image corresponding to the l-th network layer can be recorded as Here, Y0 represents a set of at least one image category. For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here.

[0187] Step S605: Update the sample category relevance using the first sample probability value and the second sample probability value.

[0188] For details, please refer to the relevant description in the aforementioned disclosed embodiment, which will not be repeated here. For the sake of convenience, the sample category correlation obtained by updating the i-th image and the j-th image obtained by the l-th network layer can be recorded as

[0189] Step S606: Using the first sample probability value and the image category annotated by the sample target image, a first loss value corresponding to the lth network layer is obtained, and using the actual category correlation and the updated sample category correlation between the sample target image and the sample reference image, a second loss value at the lth network layer is obtained.

[0190] Specifically, the cross entropy loss function (CE) can be used to calculate the probability value of the first sample. and the image category y annotated by the sample target image i , and get the first loss value corresponding to the lth network layer. For the convenience of description, it is recorded as The value range of i is from NK+1 to NK+T, that is, the first loss value is calculated only for the sample target image.

[0191] In addition, the binary cross entropy loss function (BCE) can be used to use the actual category correlation c between the sample target image and the sample reference image ij Correlation with the updated sample category The second loss value corresponding to the lth network layer is obtained. For the convenience of description, it is recorded as The value range of i is from NK+1 to NK+T, that is, the first loss value is calculated only for the sample target image.

[0192] Step S607: Determine whether the current network layer is the last network layer of the image detection model. If not, execute step S608; otherwise, execute step S609.

[0193] Step S608: Re-execute step S602 and subsequent steps.

[0194] If the current network layer is not the last network layer of the image detection model, l can be increased by 1, thereby re-executing the steps of updating the sample image features of multiple sample images based on the first network of the image detection model using the sample category correlation and subsequent steps using the network layer next to the current network layer until the current network layer is the last network layer of the image detection model. In this process, the first loss value and the second loss value corresponding to each network layer of the image detection model can be obtained.

[0195] Step S609: using the first weight corresponding to each network layer, weighting the first loss value corresponding to each network layer to obtain a first weighted loss value.

[0196] In the embodiment of the present disclosure, the later the network layer is in the image detection model, the larger the first weight corresponding to the network layer is. For the convenience of description, the first weight corresponding to the lth network layer can be recorded as For example, when l is less than L, the corresponding first weight can be set to 0.2, and when l is equal to L, the corresponding first weight can be set to 1. The specific setting can be based on actual needs. For example, based on the fact that the later the network layer is, the more important it is, the first weight corresponding to each network layer can be set to a different value, and the first weight corresponding to each network layer is greater than the first weight corresponding to the network layer before it, which is not limited here. Specifically, the first weighted loss value can be expressed as:

[0197]

[0198] Step S610: using the second weight corresponding to each network layer, weighting the second loss value corresponding to each network layer to obtain a second weighted loss value.

[0199] In the embodiment of the present disclosure, the later the network layer is in the image detection model, the larger the second weight corresponding to the network layer is. For the convenience of description, the second weight corresponding to the lth network layer can be recorded as For example, when l is less than L, the corresponding second weight can be set to 0.2, and when l is equal to L, the corresponding second weight can be set to 1. The specific setting can be based on actual needs. For example, based on the fact that the later the network layer is, the more important it is, the second weight corresponding to each network layer can be set to a different value, and the second weight corresponding to each network layer is greater than the second weight corresponding to the network layer before it, which is not limited here. Specifically, the second weighted loss value can be expressed as:

[0200]

[0201] Step S611: Adjust the network parameters of the image detection model based on the first weighted loss value and the second weighted loss value.

[0202] Specifically, the first weighted loss value and the second weighted loss value can be weighted using the weights corresponding to the first weighted loss value and the second weighted loss value, respectively, to obtain weighted loss values, and the weighted loss values ​​can be used to adjust the network parameters. Specifically, the weight corresponding to the first weighted loss value can be set to 0.5, and the weight corresponding to the second weighted loss value can also be set to 0.5, to indicate that the first weighted loss value and the second weighted loss value are equally important when adjusting the network parameters. In addition, the corresponding weights can also be adjusted according to the different importance of the first weighted loss value and the second weighted loss value, and examples will not be given one by one here.

[0203] Different from the above embodiment, the image detection model is set to include at least one sequentially connected network layer, and each network layer includes a first network and a second network. When the current network layer is not the last network layer of the image detection model, the next network layer of the current network layer is used to re-execute the first network based on the image detection model, and use the sample category correlation to update the sample image features and subsequent steps until the current network layer is the last network layer of the image detection model, thereby using the first weight corresponding to each network layer to respectively perform weighted processing on the first loss value corresponding to each network layer to obtain the first weighted loss value. Loss value, and use the second weight corresponding to each network layer to weight the second loss value corresponding to each network layer to obtain the second weighted loss value, and then adjust the network parameters of the image detection model based on the first weighted loss value and the second weighted loss value. The later the network layer is in the image detection model, the larger the first weight and the second weight corresponding to the network layer are. The loss value corresponding to the network layer of each layer of the image detection model can be obtained, and the weight corresponding to the later network layer is set to be larger, so that the data obtained by processing each network layer can be fully utilized to adjust the network parameters of image detection, which is beneficial to improving the accuracy of the image detection model.

[0204] See also Figure 7 , Figure 7 Schematic diagram of the framework of an embodiment of an image detection device 70 of the present application. Image detection device 70 includes an image acquisition module 71, a feature update module 72, and a result acquisition module 73. Image acquisition module 71 is used to acquire image features of multiple images and the category correlation of at least one set of image pairs, where the multiple images include a reference image and a target image, and each pair of images in the multiple images constitutes an image pair. The category correlation indicates the likelihood that the image pairs belong to the same image category. Feature update module 72 is used to update the image features of the multiple images using the category correlation. Result acquisition module 73 is used to use the updated image features to obtain an image category detection result for the target image.

[0205] The above scheme obtains image features of multiple images and the category correlation of at least one set of image pairs, wherein the multiple images include a reference image and a target image, and each two images in the multiple images form an image pair. The category correlation indicates the likelihood that the image pairs belong to the same image category, and the category correlation is used to update the image features, thereby using the updated image features to obtain the image category detection result of the target image. Therefore, by using the category correlation to update the image features, the image features corresponding to images of the same image category can be made closer, and the image features corresponding to images of different image categories can be made more distant, which can help improve the robustness of the image features and capture the distribution of the image features, thereby helping to improve the accuracy of image category detection.

[0206] In some disclosed embodiments, the result acquisition module 73 includes a probability prediction submodule for performing prediction processing using updated image features to obtain probability information, wherein the probability information includes a first probability value that the target image belongs to at least one reference category, and the reference category is the image category to which the reference image belongs. The result acquisition module 73 includes a result acquisition submodule for obtaining an image category detection result based on the first probability value; wherein the image category detection result is used to indicate the image category to which the target image belongs.

[0207] In some disclosed embodiments, the probability information also includes a second probability value that the reference image belongs to at least one reference category. The image detection device 70 also includes a related update module for using the probability information to update the category correlation when the number of times the prediction processing is performed meets the preset conditions, and combining with the feature update module 72 to re-execute the step of using the category correlation to update the image features. The result acquisition submodule is specifically used to obtain the image category detection result based on the first probability value when the number of times the prediction processing is performed does not meet the preset conditions.

[0208] In some disclosed embodiments, category relevance includes: a final probability value that each group of image pairs belongs to the same image category, a related update module includes an image division submodule for taking each image in a plurality of images as a current image, and taking the image pair containing the current image as a current image pair, a related update module includes a probability statistics submodule for obtaining the sum of the final probability values ​​of all current image pairs of the current image as the probability sum of the current image, a related update module includes a probability acquisition submodule for using a first probability value and a second probability value to respectively obtain a reference probability value that each group of current image pairs belongs to the same image category, and a related update module includes a probability adjustment submodule for adjusting the final probability value of each group of current image pairs using the probability sum and the reference probability value, respectively.

[0209] In some disclosed embodiments, the probability prediction submodule includes a prediction category unit for predicting the predicted categories to which the target image and the reference image belong using updated image features, wherein the predicted categories belong to at least one reference category. The probability prediction submodule includes a first matching degree acquisition unit for acquiring, for each group of image pairs, the category comparison results and feature similarities of the image pairs, and obtaining a first matching degree between the category comparison results and the feature similarities of the image pairs, wherein the category comparison results indicate whether the predicted categories to which the image pairs belong are the same, and the feature similarity indicates the similarity between the image features of the image pairs. The probability prediction submodule includes a second matching degree acquisition unit for obtaining a second matching degree between the reference image and the predicted category based on the predicted category and the reference category to which the reference image belongs. The probability prediction submodule includes a probability information acquisition unit for obtaining probability information using the first matching degree and the second matching degree.

[0210] In some disclosed embodiments, when the category comparison result is that the predicted categories are the same, the feature similarity is positively correlated with the first matching degree; when the category comparison result is that the predicted categories are different, the feature similarity is negatively correlated with the first matching degree, and the second matching degree when the predicted category is the same as the reference category is greater than the second matching degree when the predicted category is different from the reference category.

[0211] In some disclosed embodiments, the prediction category unit is specifically configured to predict the prediction category to which the image belongs based on a conditional random field network and using updated image features.

[0212] In some disclosed embodiments, the probability information acquisition unit is specifically configured to obtain the probability information based on cyclic belief propagation using the first matching degree and the second matching degree.

[0213] In some disclosed embodiments, the preset condition includes: the number of times the prediction process is performed does not reach a preset threshold.

[0214] In some disclosed embodiments, the step of updating image features using category relevance is performed by a graph neural network.

[0215] In some disclosed embodiments, the feature update module 72 includes a feature acquisition submodule for obtaining intra-class image features and inter-class image features using category correlation and image features. The feature update module 72 includes a feature conversion submodule for performing feature conversion using intra-class image features and inter-class image features to obtain updated image features.

[0216] In some disclosed embodiments, the image detection device 70 also includes an initialization module, which is specifically used to determine the initial category correlation of the image pair as a preset upper limit value when the image pair belongs to the same image category; determine the initial category correlation of the image pair as a preset lower limit value when the image pair belongs to different image categories; and determine the initial category correlation of the image pair as a preset value between the preset lower limit value and the preset upper limit value when at least one of the image pairs is a target image.

[0217] See also Figure 8 , Figure 8 The present invention is a schematic diagram of a framework of an embodiment of a training device 80 for an image detection model of the present invention. The training device 80 for an image detection model includes a sample acquisition module 81, a feature update module 82, a result acquisition module 83, and a parameter adjustment module 84. The sample acquisition module 81 is used for sample image features of multiple sample images and sample category correlation of at least one set of sample image pairs, wherein the multiple sample images include a sample reference image and a sample target image, and every two sample images in the multiple sample images form a set of sample image pairs, and the sample category correlation indicates the possibility that the sample image pairs belong to the same image category. The feature update module 82 is used to update the sample image features of the multiple sample images based on the first network of the image detection model using the sample category correlation. The result acquisition module 83 is used to obtain image category detection results of the sample target images based on the second network of the image detection model using the updated sample image features. The parameter update module 84 is used to adjust the network parameters of the image detection model using the image category detection results of the sample target images and the image categories annotated on the sample target images.

[0218] The above scheme obtains sample image features of multiple sample images and sample category correlation of at least one group of sample image pairs, and the multiple sample images include sample reference images and sample target images, and every two sample images in the multiple sample images form a group of sample image pairs. The sample category correlation indicates the possibility that the sample image pairs belong to the same image category, and based on the first network of the image detection model, the sample image features of the multiple sample images are updated using the sample category correlation, so that based on the second network of the image detection model, the image category detection results of the sample target images are obtained using the updated sample image features, and then the network parameters of the image detection model are adjusted using the image category detection results and the image categories annotated by the sample target images. Therefore, by updating the sample image features using the sample category correlation, the sample image features corresponding to images of the same image category can be made closer, and the sample image features corresponding to images of different image categories can be made more distant, which can be beneficial to improving the robustness of the sample image features, and can be beneficial to capturing the distribution of the sample image features, which can be beneficial to improving the accuracy of the image detection model.

[0219] In some disclosed embodiments, the result acquisition module 83 includes a probability information acquisition submodule for performing prediction processing based on the second network using the updated sample image features to obtain sample probability information, wherein the sample probability information includes a first sample probability value that the sample target image belongs to at least one reference category and a second sample probability value that the sample reference image belongs to at least one reference category, and the reference category is the image category to which the sample reference image belongs. The result acquisition module 83 includes a detection result acquisition submodule for obtaining an image category detection result of the sample target image based on the first sample probability value. The image detection model training device 80 also includes a correlation update module for updating the sample category correlation using the first sample probability value and the second sample probability value. The parameter update module 84 includes a first loss calculation submodule for obtaining a first loss value of the image detection model using the first sample probability value and the image category annotated by the sample target image. The parameter update module 84 includes a second loss calculation submodule for obtaining a second loss value of the image detection model using the actual category correlation and the updated sample category correlation between the sample target image and the sample reference image. The parameter update module 84 includes a parameter adjustment submodule for adjusting the network parameters of the image detection model based on the first loss value and the second loss value.

[0220] In some disclosed embodiments, the image detection model includes at least one sequentially connected network layer, each network layer includes a first network and a second network, and the feature update module 82 is also used to use the next network layer of the current network layer to re-execute the steps of updating the sample image features based on the first network of the image detection model and the sample category correlation and subsequent steps when the current network layer is not the last network layer of the image detection model, until the current network layer is the last network layer of the image detection model, the parameter adjustment submodule includes a first weighting unit, which is used to use the first weight corresponding to each network layer to weight the first loss value corresponding to each network layer to obtain a first weighted loss value, the parameter adjustment submodule includes a second weighting unit, which is used to use the second weight corresponding to each network layer to weight the second loss value corresponding to each network layer to obtain a second weighted loss value, and the parameter adjustment submodule includes a parameter adjustment unit, which is used to adjust the network parameters of the image detection model based on the first weighted loss value and the second weighted loss value, wherein the later the network layer is in the image detection model, the larger the first weight and the second weight corresponding to the network layer are.

[0221] See also Figure 9 , Figure 9: is a schematic diagram of a framework of an embodiment of an electronic device 90 of the present application. The electronic device 90 includes a memory 91 and a processor 92 coupled to each other. The processor 92 is used to execute program instructions stored in the memory 91 to implement the steps in any of the above-mentioned image detection method embodiments, or to implement the steps in any of the above-mentioned image detection model training method embodiments. In a specific implementation scenario, the electronic device 90 may include but is not limited to: a microcomputer, a server. In addition, the electronic device 90 may also include a mobile device such as a laptop computer and a tablet computer, or the electronic device 90 may also be a surveillance camera, etc., which is not limited here.

[0222] Specifically, the processor 92 is used to control itself and the memory 91 to implement the steps in any of the above-mentioned image detection method embodiments, or to implement the steps in the training method embodiment of any of the above-mentioned image detection models. The processor 92 can also be referred to as a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip with signal processing capabilities. The processor 92 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 92 can be implemented by an integrated circuit chip.

[0223] The above solution can improve the accuracy of image category detection.

[0224] See also Figure 10 , Figure 10 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 100 of the present application. The computer-readable storage medium 100 stores program instructions 101 that can be executed by a processor. The program instructions 101 are used to implement the steps of any of the above-mentioned image detection method embodiments, or to implement the steps of any of the above-mentioned image detection model training method embodiments.

[0225] The above solution can improve the accuracy of image category detection.

[0226] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0227] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0228] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0229] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0230] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0231] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. An image detection method, characterized in that: include: Obtaining image features of a plurality of images and category correlations of at least one set of image pairs; wherein the plurality of images include a reference image and a target image, and each two images in the plurality of images constitute a set of the image pairs; the category correlations represent the likelihood that the image pairs belong to the same image category; when the image pairs belong to the same image category, determining an initial category correlation of the image pairs as a preset upper limit value; when the image pairs belong to different image categories, determining an initial category correlation of the image pairs as a preset lower limit value; and when at least one of the image pairs is the target image, determining the initial category correlation of the image pairs as a preset value between the preset lower limit value and the preset upper limit value; Updating the image features of the plurality of images using the category relevance; Obtaining an image category detection result of the target image using the updated image features, wherein the image category detection result is used to indicate the image category to which the target image belongs, and obtaining the image category detection result of the target image using the updated image features includes: performing prediction processing using the updated image features to obtain probability information, wherein the probability information includes a first probability value that the target image belongs to at least one reference category and a second probability value that the reference image belongs to the at least one reference category, the reference category being the image category to which the reference image belongs; When the number of times the prediction process is performed satisfies a preset condition, updating the category relevance using the probability information, and re-performing the step of updating the image features of the plurality of images using the category relevance; wherein the preset condition includes: the number of times the prediction process is performed does not reach a preset threshold; When the number of times the prediction process is performed does not satisfy the preset condition, the image category detection result is obtained based on the first probability value.

2. The method according to claim 1, characterized in that The category relevance includes: a final probability value of each group of image pairs belonging to the same image category; and updating the category relevance using the probability information includes: Taking each of the multiple images as a current image, and taking the image pair including the current image as a current image pair; Obtaining the sum of the final probability values ​​of all the current image pairs of the current image as the probability sum of the current image; and Using the first probability value and the second probability value, respectively obtaining a reference probability value that each group of the current image pairs belongs to the same image category; The final probability value of each group of the current image pairs is adjusted using the probability sum and the reference probability value respectively.

3. The method according to claim 1 or 2, characterized in that The method of performing prediction processing using the updated image features to obtain probability information includes: using the updated image features to predict a predicted category to which the image belongs, wherein the predicted category belongs to the at least one reference category; For each group of the image pairs, obtaining a category comparison result and a feature similarity of the image pairs, and obtaining a first matching degree between the category comparison result and the feature similarity of the image pairs; wherein the category comparison result indicates whether the predicted categories to which the image pairs belong are the same, and the feature similarity indicates the similarity between the image features of the image pairs; and obtaining a second matching degree of the reference image with respect to the predicted category and the reference category based on the predicted category to which the reference image belongs and the reference category; The probability information is obtained using the first matching degree and the second matching degree.

4. The method according to claim 3, characterized in that When the category comparison result shows that the predicted categories are the same, the feature similarity is positively correlated with the first matching degree; when the category comparison result shows that the predicted categories are different, the feature similarity is negatively correlated with the first matching degree, and the second matching degree when the predicted category is the same as the reference category is greater than the second matching degree when the predicted category is different from the reference category.

5. The method according to claim 3, characterized in that The predicting the prediction category to which the image belongs by using the updated image features includes: Based on the conditional random field network, the updated image features are used to predict the prediction category to which the image belongs.

6. The method according to claim 3, characterized in that The obtaining the probability information by using the first matching degree and the second matching degree includes: Based on cyclic belief propagation, the probability information is obtained using the first matching degree and the second matching degree.

7. The method according to claim 1, characterized in that The step of updating the image features of the multiple images using the category relevance is performed by a graph neural network.

8. The method according to claim 1, characterized in that The updating of the image features of the plurality of images by using the category relevance includes: Obtaining intra-class image features and inter-class image features using the class correlation and the image features; Feature conversion is performed using the intra-class image features and the inter-class image features to obtain updated image features.

9. A training method for an image detection model, characterized in that: The image detection model is applied to the image detection method according to any one of claims 1 to 8, and the training method of the image detection model includes: Obtaining sample image features of a plurality of sample images and sample category correlation of at least one set of sample image pairs; wherein the plurality of sample images include a sample reference image and a sample target image, each two sample images in the plurality of sample images form a set of the sample image pairs, and the sample category correlation indicates the likelihood that the sample image pairs belong to the same image category; Based on the first network of the image detection model, using the sample category correlation, updating the sample image features of the plurality of sample images; Based on the second network of the image detection model, using the updated sample image features, obtaining an image category detection result of the sample target image; wherein the image category detection result is used to indicate the image category to which the sample target image belongs; The network parameters of the image detection model are adjusted using the image category detection result of the sample target image and the image category annotated by the sample target image.

10. The method according to claim 9, characterized in that The second network based on the image detection model obtains an image category detection result of the sample target image using the updated sample image features, including: performing prediction processing based on the second network using the updated sample image features to obtain sample probability information, wherein the sample probability information includes a first sample probability value that the sample target image belongs to at least one reference category and a second sample probability value that the sample reference image belongs to the at least one reference category, where the reference category is the image category to which the sample reference image belongs; Obtaining an image category detection result of the sample target image based on the first sample probability value; Before adjusting the network parameters of the image detection model using the image category detection result of the sample target image and the image category annotated by the sample target image, the method further includes: Updating the sample category correlation using the first sample probability value and the second sample probability value; The adjusting the network parameters of the image detection model using the image category detection result of the sample target image and the image category annotated by the sample target image includes: Obtaining a first loss value of the image detection model using the first sample probability value and the image category annotated by the sample target image; and Obtaining a second loss value of the image detection model using the actual category correlation and the updated sample category correlation between the sample target image and the sample reference image; Based on the first loss value and the second loss value, adjust the network parameters of the image detection model.

11. The method according to claim 10, characterized in that The image detection model includes at least one sequentially connected network layer, each of the network layers includes one first network and one second network; Before adjusting the network parameters of the image detection model based on the first loss value and the second loss value, the method further includes: When the current network layer is not the last network layer of the image detection model, re-execute the step of updating the sample image features of the plurality of sample images based on the first network of the image detection model and using the sample category correlation using the next network layer of the current network layer, and subsequent steps until the current network layer is the last network layer of the image detection model; The adjusting the network parameters of the image detection model based on the first loss value and the second loss value includes: Using the first weights corresponding to the respective network layers, the first loss values ​​corresponding to the respective network layers are weighted to obtain first weighted loss values; and Using the second weights corresponding to the respective network layers, the second loss values ​​corresponding to the respective network layers are weighted to obtain second weighted loss values; Adjusting network parameters of the image detection model based on the first weighted loss value and the second weighted loss value; The later the network layer is in the image detection model, the larger the first weight and the second weight corresponding to the network layer are.

12. An image detection device, characterized in that: include: an image acquisition module, configured to acquire image features of a plurality of images and category correlations of at least one set of image pairs; wherein the plurality of images include a reference image and a target image, and each two images in the plurality of images form a set of image pairs; the category correlations represent the likelihood that the image pairs belong to the same image category; when the image pairs belong to the same image category, the initial category correlations of the image pairs are determined to be a preset upper limit value; when the image pairs belong to different image categories, the initial category correlations of the image pairs are determined to be a preset lower limit value; and when at least one of the image pairs is the target image, the initial category correlations of the image pairs are determined to be a preset value between the preset lower limit value and the preset upper limit value; a feature updating module, configured to update the image features of the plurality of images using the category correlation; A result acquisition module is configured to obtain an image category detection result of the target image using the updated image features; wherein the image category detection result is used to indicate the image category to which the target image belongs, and obtaining the image category detection result of the target image using the updated image features includes: performing prediction processing using the updated image features to obtain probability information, wherein the probability information includes a first probability value that the target image belongs to at least one reference category and a second probability value that the reference image belongs to the at least one reference category, the reference category being the image category to which the reference image belongs; When the number of times the prediction process is performed satisfies a preset condition, updating the category relevance using the probability information, and re-performing the step of updating the image features of the plurality of images using the category relevance; wherein the preset condition includes: the number of times the prediction process is performed does not reach a preset threshold; When the number of times the prediction process is performed does not satisfy the preset condition, the image category detection result is obtained based on the first probability value.

13. A training device for an image detection model, characterized in that: The image detection model is applied to the image detection device according to claim 12, and the training device of the image detection model includes: a sample acquisition module, configured to acquire sample image features of a plurality of sample images and a sample category correlation of at least one set of sample image pairs; wherein the plurality of sample images include a sample reference image and a sample target image, and every two sample images in the plurality of sample images form a set of the sample image pairs; and the sample category correlation indicates the likelihood that the sample image pairs belong to the same image category; a feature updating module, configured to update the sample image features of the plurality of sample images based on the first network of the image detection model and using the sample category correlation; A result acquisition module is configured to obtain an image category detection result of the sample target image based on the second network of the image detection model and using the updated sample image features; wherein the image category detection result is used to indicate the image category to which the sample target image belongs; The parameter updating module is used to adjust the network parameters of the image detection model by using the image category detection result of the sample target image and the image category annotated by the sample target image.

14. An electronic device, characterized in that: It includes a memory and a processor coupled to each other, and the processor is used to execute program instructions stored in the memory to implement the image detection method described in any one of claims 1 to 8, or the training method of the image detection model described in any one of claims 9 to 11.

15. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by the processor, the image detection method described in any one of claims 1 to 8, or the image detection model training method described in any one of claims 9 to 11 is implemented.

Images