Image detection method and device, electronic equipment, storage medium and product

Abstract

The invention provides an image detection method and device, electronic equipment, a storage medium and a product, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first to-be-detected image and a trained classification model; the classification model is trained by using a sample image and a label representing a real category; extracting features of the second to-be-detected image by using a feature extraction network to obtain first image features; the second to-be-detected image is obtained based on the first to-be-detected image; classifying the second image features by using a linear classification network to obtain first prediction probability distribution; the second image feature is obtained based on the first image feature; calculating a first difference value representing a difference between the first predicted probability distribution and the reference probability distribution; if the second difference value is not greater than the first preset threshold value, determining that the object in the first to-be-detected image belongs to an unknown category; the second difference value is determined based on the first difference value. According to the invention, the accuracy of image detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image detection method, device, electronic equipment, storage medium and product. Background Art

[0002] The classification model to be trained is trained using sample images and the true categories of objects in the sample images to obtain a trained classification model. The trained classification model can accurately identify categories (that is, known categories) including: the true categories of objects in the sample images used during training.

[0003] However, images of unknown categories are not involved in training, and the classification model has a weak ability to recognize images of unknown categories. If an image of unknown category is input into the classification model, the classification model cannot clearly determine which known category the object in the image of unknown category belongs to. The probability of the object in the output image belonging to each known category is close, and the classification model will determine the known category with a higher probability as the category of the object in the input image. In other words, the classification model will misjudge the unknown category as a known category. Therefore, there is an urgent need for a method to distinguish objects of known and unknown categories to improve the accuracy of image detection. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an image detection method, device, electronic device, storage medium and product to improve the accuracy of image detection. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides an image detection method, the method comprising:

[0006] Acquire a first image to be detected and a trained classification model; wherein the acquired classification model is obtained by training using each sample image and a label representing the true category of the object in the sample image; the classification model includes a feature extraction network and a linear classification network;

[0007] Using the feature extraction network to extract features from a second image to be detected, to obtain features of the first image; wherein the second image to be detected is obtained based on the first image to be detected;

[0008] The second image feature is classified by using the linear classification network to obtain a first predicted probability distribution; wherein the first predicted probability distribution includes: the predicted probability that the object in the second to-be-detected image belongs to each preset category; each preset category includes the real category of the object in the sample image; the second image feature is obtained based on the first image feature;

[0009] Calculating a first difference value for indicating a difference between the first predicted probability distribution and a reference probability distribution; wherein the probabilities corresponding to the preset categories in the reference probability distribution are the same;

[0010] If the second difference value is not greater than the first preset threshold, it is determined that the object in the first to-be-detected image belongs to an unknown category; wherein the second difference value is determined based on the calculated first difference value.

[0011] Optionally, before using the linear classification network to classify the second image feature to obtain the first predicted probability distribution, the method further includes:

[0012] For each element in the first image feature, if the value of the element is greater than a second preset threshold, the value of the element is adjusted to the second preset threshold to obtain a second image feature; wherein the second preset threshold is determined based on the elements in the image feature obtained by extracting features from the sample image using the feature extraction network.

[0013] Optionally, a method for determining the second preset threshold includes:

[0014] Selecting at least one sample image from all sample images used to train the classification model;

[0015] Inputting each sample image among the selected sample images into the feature extraction network of the trained classification model to obtain a third image feature of the sample image;

[0016] Based on the target value obtained in each third image feature, a second preset threshold is determined; wherein, for the elements in each third image feature, the proportion of elements greater than the target value in the third image feature to each element in the third image feature is a specified proportion.

[0017] Optionally, determining the second preset threshold based on the obtained target value in each third image feature includes:

[0018] If there are multiple sample images selected, the average value of the sum of the target values ​​in the multiple third image features is taken as the second preset threshold;

[0019] If only one sample image is selected, the target value in the third image feature of the sample image is obtained as the second preset threshold.

[0020] Optional,

[0021] The second image to be detected is the same as the first image to be detected, and the first difference value is the same as the second difference value;

[0022] or,

[0023] The second image to be detected includes: multiple images obtained by performing specified image processing on the first image to be detected; the specified image processing includes at least one of the following: cropping, flipping; before determining that the object in the first image to be detected belongs to an unknown category if the second difference value is greater than the first preset threshold, it also includes: calculating the average value of the sum of each first difference value obtained based on each second image to be detected to obtain a second difference value.

[0024] Optionally, the calculating a first difference value representing a difference between the first predicted probability distribution and a reference probability distribution includes:

[0025] Calculating, by using a Manhattan norm function, a first vector norm of the second image feature and a second vector norm of a difference between a predicted probability distribution and a reference probability distribution of the second image to be detected;

[0026] The first vector norm, the second vector norm, the number of dimensions of the second image feature, and the total number of the preset categories are calculated according to a specified formula to determine a first difference value; wherein the specified formula is: S represents the first difference value, U represents the first vector norm, V represents the second vector norm, feat_dim represents the number of dimensions of the second image feature, and num_classes represents the total number of the preset categories.

[0027] Optionally, when the second image to be detected includes multiple images, the method further includes:

[0028] If the second difference value is greater than the first preset threshold, for each preset category, taking the average value of the sum of the predicted probabilities of the preset category in each of the first predicted probability distributions as the predicted probability of the preset category, to obtain a second predicted probability distribution;

[0029] The preset category with the largest predicted probability in the second predicted probability distribution is used as the category of the object in the first image to be detected.

[0030] In a second aspect, an embodiment of the present invention provides an image detection device, the device comprising:

[0031] A first acquisition module is used to acquire a first image to be detected and a trained classification model; wherein the acquired classification model is obtained by training using each sample image and a label representing the true category of the object in the sample image; the classification model includes a feature extraction network and a linear classification network;

[0032] A first utilizing module is used to utilize the feature extraction network to perform feature extraction on a second image to be detected to obtain first image features; wherein the second image to be detected is obtained based on the first image to be detected;

[0033] A second utilization module is used to classify the second image feature using the linear classification network to obtain a first predicted probability distribution; wherein the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category; each preset category includes the real category of the object in the sample image; the second image feature is obtained based on the first image feature;

[0034] A first calculation module, used to calculate a first difference value used to represent the difference between the first predicted probability distribution and a reference probability distribution; wherein the probabilities corresponding to the preset categories in the reference probability distribution are the same;

[0035] The first determination module is used to determine that the object in the first to-be-detected image belongs to an unknown category if the second difference value is not greater than a first preset threshold; wherein the second difference value is determined based on the calculated first difference value.

[0036] An embodiment of the present invention further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0037] Memory, used to store computer programs;

[0038] The processor is used to implement the image detection method when executing the program stored in the memory.

[0039] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image detection method is implemented.

[0040] An embodiment of the present invention further provides a computer program product, including a computer program, and the computer program implements the image detection method when executed by a processor.

[0041] Beneficial effects of the embodiments of the present invention:

[0042] An image detection method provided by an embodiment of the present invention can obtain a first image to be detected and a trained classification model; the classification model includes a feature extraction network and a linear classification network; the feature extraction network is used to extract features of a second image to be detected to obtain first image features; the second image to be detected is obtained based on the first image to be detected; the second image features are classified using a linear classification network to obtain a first predicted probability distribution; the second image features are obtained based on the first image features; a first difference value used to represent the difference between the first predicted probability distribution and the reference probability distribution is calculated; if the second difference value is not greater than a first preset threshold, it is determined that the object in the first image to be detected belongs to an unknown category; the second difference value is determined based on the calculated first difference value.

[0043] Based on the above processing, the trained classification model can accurately identify images of real categories (that is, known categories). Since the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category, and each preset category includes the real category of the object in the sample image; thus, the first predicted probability distribution represents the predicted probability that the object in the second image to be detected belongs to each known category. Since the predicted probabilities of belonging to each preset category output by the classification model for images of unknown categories are close, the predicted probability distribution output by the classification model for images of unknown categories is relatively uniform. The corresponding probabilities of each preset category in the reference probability distribution are the same, and the predicted probability output by the classification model for images of unknown categories can be simulated by the reference probability distribution. If the first difference value representing the difference between the first predicted probability distribution and the reference probability distribution is smaller, it means that the closer the first predicted probability distribution is to the reference probability distribution, the higher the possibility that the object in the second image to be detected is an unknown category; further, if the second difference value determined based on the first difference value is not greater than the first preset threshold, it can be determined that the object in the first image to be detected belongs to an unknown category. It can be seen that the image detection method provided by the embodiment of the present invention can distinguish objects of known categories from objects of unknown categories to improve the accuracy of image detection.

[0044] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0046] Figure 1 A schematic diagram of a flow chart of a first image detection method provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a flow chart of a second image detection method provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of a flow chart of a third image detection method provided by an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a fourth image detection method provided by an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of the principle of an image detection method provided by an embodiment of the present invention;

[0051] Figure 6 A schematic diagram of the structure of an image detection device provided by an embodiment of the present invention;

[0052] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0054] If the category of the image input into the trained classification model is a new category (unknown category), and the image of the unknown category does not participate in the training of the classification model, then the classification model will randomly determine the object in the image of the unknown category as an object of a known category. Among them, the image of the unknown category does not participate in the training of the classification model, belongs to the data outside the training data, and can also be called the out-of-distribution sample (OOD); the image of the known category participates in the training of the classification model, belongs to the training data, and can also be called the in-distribution sample (ID).

[0055] However, images of unknown categories are not involved in training, and the classification model has a weak ability to recognize images of unknown categories. If an image of unknown category is input into the classification model, the probability of the object in the image belonging to each known category output by the classification model is close, and the classification model will determine the known category with a higher probability as the category of the object in the input image. In other words, the classification model will mistakenly judge the unknown category as a known category. Therefore, a method is urgently needed to distinguish objects of known and unknown categories to improve the accuracy of image detection.

[0056] In order to improve the accuracy of image detection, the present invention provides an image detection method, device, electronic equipment, storage medium and product.

[0057] Among them, an image detection method provided by an embodiment of the present invention can be applied to an electronic device. The electronic device can obtain an image to be detected and perform image detection processing; in a specific application, the electronic device can be a server or a terminal device, and the embodiment of the present invention is only used as an example for illustration and is not specifically limited.

[0058] An image detection method provided by an embodiment of the present invention may include the following steps:

[0059] Obtaining a first image to be detected and a trained classification model; wherein the obtained classification model is obtained by training using each sample image and a label representing the true category of the object in the sample image; the classification model includes a feature extraction network and a linear classification network;

[0060] Using a feature extraction network to extract features from a second image to be detected, to obtain features of the first image; wherein the second image to be detected is obtained based on the first image to be detected;

[0061] The second image feature is classified using a linear classification network to obtain a first prediction probability distribution; wherein the first prediction probability distribution includes: a prediction probability that an object in the second image to be detected belongs to each preset category; each preset category includes a real category of the object in the sample image; the second image feature is obtained based on the first image feature;

[0062] Calculating a first difference value for indicating a difference between the first predicted probability distribution and a reference probability distribution; wherein the probabilities corresponding to the preset categories in the reference probability distribution are the same;

[0063] If the second difference value is not greater than the first preset threshold, it is determined that the object in the first to-be-detected image belongs to an unknown category; wherein the second difference value is determined based on the calculated first difference value.

[0064] Based on the above processing, the trained classification model can accurately identify images of real categories (that is, known categories). Since the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category, and each preset category includes the real category of the object in the sample image; thus, the first predicted probability distribution represents the predicted probability that the object in the second image to be detected belongs to each known category. Since the predicted probabilities of belonging to each preset category output by the classification model for images of unknown categories are close, the predicted probability distribution output by the classification model for images of unknown categories is relatively uniform. The corresponding probabilities of each preset category in the reference probability distribution are the same, and the predicted probability output by the classification model for images of unknown categories can be simulated by the reference probability distribution. If the first difference value representing the difference between the first predicted probability distribution and the reference probability distribution is smaller, it means that the closer the first predicted probability distribution is to the reference probability distribution, the higher the possibility that the object in the second image to be detected is an unknown category; further, if the second difference value determined based on the first difference value is not greater than the first preset threshold, it can be determined that the object in the first image to be detected belongs to an unknown category. It can be seen that the image detection method provided by the embodiment of the present invention can distinguish objects of known categories from objects of unknown categories to improve the accuracy of image detection.

[0065] The following describes an image detection method provided by an embodiment of the present invention in conjunction with the accompanying drawings. Figure 1 , Figure 1 This is a first flow chart of an image detection method provided by an embodiment of the present invention. The method may include steps S101-S105.

[0066] S101, obtaining a first image to be detected and a trained classification model.

[0067] The obtained classification model is obtained by training with each sample image and a label representing the true category of the object in the sample image; the classification model includes a feature extraction network and a linear classification network.

[0068] It is understandable that the electronic device can obtain the first image to be detected from a storage server or a local storage space storing the first image to be detected. The category of the object in the obtained first image to be detected can be a known category or an unknown category, and the electronic device cannot determine whether it is a known category or an unknown category. The categories involved in the embodiments of the present invention are all categories of objects in the image, and the categories of the images described in the subsequent embodiments also indicate the categories of objects in the image.

[0069] The electronic device can obtain the network parameters of the trained classification model, and the obtained network parameters can be used to process the image data in the subsequent steps. The classification model is trained using sample images and labels representing the real categories of objects in the sample images, so that the trained classification model can accurately identify objects of real categories. For example, the classification model is trained based on images of categories A, B, C, and D, and categories A, B, C, and D are all real categories. When the image of category A is input into the classification model, it can be obtained that the probability that the object in the image belongs to category A is 95%, the probability that it belongs to category B is 3%, the probability that it belongs to category C is 1%, and the probability that it belongs to category D is 1%, thereby determining that the category of the object in the input image is category A.

[0070] Among them, the classification model can perform classification processing on the input image. The classification model can include a feature extraction network and a linear classification network. The feature extraction network can extract image features of the input image, and the linear classification network can classify the image features extracted by the feature extraction layer to obtain classified image features.

[0071] Optionally, the classification model can be a Data-efficient Image Transformer-B (DeiT-B) model, which is an image classification model based on a Vision Transformer (ViT). The DeiT-B model has a feature extraction network and a linear classification network. The DeiT-B model is completely based on the Transformer architecture and can handle the global dependencies between different features. Thus, technicians can train the DeiT-B model with a smaller amount of data to obtain a classification model with excellent performance.

[0072] Optionally, in order to ensure that the classification model has the same recognition ability for images of each real category, the number of sample images of different real categories is the same when training the classification model. For example, the classification model can be trained using 200 sample images containing objects belonging to category A, 200 sample images containing objects belonging to category B, 200 sample images containing objects belonging to category C, and 200 sample images containing objects belonging to category D. In this case, the proportion of sample images belonging to each real category in all sample images is the same.

[0073] S102, using a feature extraction network to extract features from a second image to be detected to obtain first image features.

[0074] The second image to be detected is obtained based on the first image to be detected.

[0075] It is understandable that before feature extraction, a second image to be detected can be obtained based on the first image to be detected. The category of the object in the second image to be detected is the same as the category of the object in the first image to be detected. Optionally, in one implementation, the first image to be detected can be enhanced to obtain a second image to be detected with high definition to improve the accuracy of the feature vector extracted subsequently; in another implementation, the first image to be detected can be processed to reduce the definition to obtain a second image to be detected with original image quality and a second image to be detected with low definition, so as to increase the number of reference images, enrich the detection results, and improve the detection accuracy. In this regard, the embodiments of the present invention are only used as examples for illustration and are not specifically limited.

[0076] The feature extraction network in the acquired classification model can be used to extract features from the second detection image. Specifically, the second image to be detected is processed using the network parameters of the feature extraction network to obtain the first image feature. The first image feature can be represented as a feature vector.

[0077] S103, classify the second image features using a linear classification network to obtain a first predicted probability distribution.

[0078] Among them, the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category; each preset category includes the real category of the object in the sample image; the second image feature is obtained based on the first image feature.

[0079] It is understandable that before classification, the second image feature can be obtained based on the first image feature. Optionally, the first image feature can be mapped to obtain the second image feature to reduce the dimension and the amount of image data. This embodiment of the present invention is only used as an example and is not specifically limited.

[0080] The network parameters of the linear classification network can be used to classify the second image features to obtain the classified image features; the first predicted probability distribution can be obtained through the classified image features. Exemplarily, the classification model also includes an output network, and the classified image features are mapped through the network parameters of the output network to obtain the mapped image features. The mapped image features can be normalized using the activation function in the output network to obtain the predicted probability that the object in the second image to be detected belongs to each preset category. Among them, the activation function can be a normalized exponential (Softmax) function, and the network parameters of the output network can be in the form of a weight matrix containing each network parameter.

[0081] The first predicted probability distribution includes: the predicted probability that the object in the second to-be-detected image belongs to each preset category. For example, each preset category includes: category A, category B, category C, category D, and the first predicted probability distribution may include: the probability that the object belongs to category A, the probability that the object belongs to category B, the probability that the object belongs to category C, and the probability that the object belongs to D.

[0082] Each preset category includes the real category of the object in the sample image. The classification model can be trained using sample images of multiple real categories. The trained classification model can identify the predicted probability that the object in the input image belongs to each real category. Among them, the preset category is the real category, that is, the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each real category.

[0083] S104, calculating a first difference value used to represent the difference between the first predicted probability distribution and the reference probability distribution.

[0084] Among them, the probabilities corresponding to the preset categories in the reference probability distribution are the same.

[0085] It is understandable that the reference probability distribution is a uniform distribution with the same probability corresponding to each preset category. For example, if the classification model is trained based on images of categories A, B, C, and D, the reference probability distribution may include: the probability that the object belongs to category A is 25%, the probability that the object belongs to category B is 25%, the probability that the object belongs to category C is 25%, and the probability that the object belongs to category D is 25%.

[0086] The probability in the reference probability distribution can be used as the true probability that the object belongs to each preset category when the classification model detects an object of an unknown category, and the difference between the predicted probability and the true probability that the image to be detected belongs to each preset category can be compared. Specifically, a first difference value representing the difference between the predicted probability of each preset category in the first predicted probability distribution and the probability of the same preset category in the reference probability distribution can be calculated.

[0087] Optionally, in one implementation, step S104 may include steps S1041 - S1042 .

[0088] S1041, using the Manhattan norm function, calculating a first vector norm of the second image feature and a second vector norm of the difference between the predicted probability distribution and the reference probability distribution of the second image to be detected.

[0089] S1042: Calculate the first vector norm, the second vector norm, the number of dimensions of the second image feature, and the total number of preset categories according to a specified formula to determine a first difference value.

[0090] The specified formula is: S represents the first difference value, U represents the first vector norm, V represents the second vector norm, feat_dim represents the number of dimensions of the second image feature, and num_classes represents the total number of preset categories.

[0091] It can be understood that the Manhattan norm function can be used to calculate the vector norm (i.e., the first vector norm) of the second image feature. That is, the Manhattan norm function can be used to calculate the sum of the absolute values ​​of each element in the second image feature to obtain the first vector norm. For each preset category, the difference between the predicted probability of the preset category in the first predicted probability distribution of the second image to be detected and the true probability of the preset category in the reference probability distribution can be calculated. Specifically, the first predicted probability distribution of the second image to be detected can be expressed in vector form, and each element in the vector representing the first predicted probability distribution of the second image to be detected can represent: the predicted probability of belonging to a preset category; similarly, the reference probability distribution can be expressed in vector form, and each element in the vector representing the reference probability distribution can represent: the true probability of belonging to a preset category. The vector difference between the vector representing the first predicted probability distribution of the second image to be detected and the vector representing the reference probability distribution can be calculated; then the Manhattan norm function is used to calculate the sum of the absolute values ​​of each element in the vector difference of the two vectors to obtain the second vector norm.

[0092] Then, the first vector norm, the second vector norm, the number of dimensions of the second image feature, and the total number of preset categories may be calculated according to a specified formula to determine a first difference value.

[0093] In an embodiment of the present invention, if a convolutional neural network model is used to extract image features, the elements in the second image features can be any real numbers, which may include negative numbers; and the difference between the predicted probability distribution of the second image to be detected and the reference probability distribution may be positive or negative. The Manhattan norm function can be used to directly calculate the sum of the absolute values ​​of the elements in the second image features and the sum of the absolute values ​​of the difference between the predicted probability distribution and the reference probability distribution, which can intuitively reflect the probability difference between each preset category in the predicted probability distribution and the reference probability distribution, thereby improving the effectiveness of the first difference value.

[0094] Optionally, in another implementation, Kullback-Leibler (KL) divergence may measure the difference between two probability distributions. The smaller the KL divergence is, the closer the two probability distributions are.

[0095] For example, the first difference value may be determined by calculating the KL divergence between the first predicted probability distribution and the reference probability distribution.

[0096] Exemplarily, the KL divergence between the first predicted probability distribution and the reference probability distribution may be calculated according to formula (1).

[0097]

[0098] Among them, D KL (P‖Q) represents the KL divergence between the first predicted probability distribution and the reference probability distribution, P represents the reference probability distribution, p i represents the true probability of the i-th preset category in the reference probability distribution, Q represents the first predicted probability distribution, q i represents the predicted probability of the i-th preset category in the first predicted probability distribution, the maximum value of i is the total number of preset categories, H(P,Q) represents the cross entropy between the first predicted probability distribution and the reference probability distribution, and H(P) represents the entropy of the reference probability distribution.

[0099] The reference probability distribution is a uniform distribution with the same probability corresponding to each preset category. For example, the reference probability distribution can be a uniform distribution Where u represents the reference probability distribution, represents the reference probability distribution as a C-dimensional vector space, where C represents the number of preset categories. Represents the probability of any preset category in the reference probability distribution.

[0100] q in formula (1) i It can be represented by the softmax function output of the classification model, that is, Substituting the uniform distribution as the reference probability distribution into the formula (1) for calculating the KL divergence, we can obtain formula (2):

[0101]

[0102] Among them, D KL (u‖Softmax(f(x)) represents the KL divergence between the first predicted probability distribution and the reference probability distribution when the reference probability distribution is uniformly distributed; T represents the temperature coefficient, which is a hyperparameter; f c (x) represents the second image feature, f i (x) represents the image feature belonging to the i-th preset category; H(u) represents the entropy of the reference probability distribution when the reference probability distribution is a uniform distribution; and H(u) is usually a constant.

[0103] The KL divergence can be processed by the gradient normalization (GradNorm) method to calculate the parameter gradient of the KL divergence in the direction of each network parameter of the classification model. The parameter gradient can indicate the rate of change of the KL divergence in the parameter space. A smaller parameter gradient means that the KL divergence changes little, the gap between the first predicted probability distribution and the reference probability distribution is small, and the object in the image to be detected is likely to belong to an unknown category.

[0104] Since the parameter gradient can also indicate the direction of change of the KL divergence in the parameter space, the parameter gradient may be positive or negative. In order to directly reflect the magnitude of the KL divergence change, the vector norm of each parameter gradient can be calculated using the Manhattan norm function.

[0105] For example, the KL divergence can be processed by combining the gradient normalization method with the vector norm through formula (3), and the formula (3) is as follows:

[0106]

[0107] Where S(x) represents the first difference value, θ represents each network parameter in the classification model, ‖·‖ p Indicates L p norm, also known as the Manhattan norm.

[0108] S105: If the second difference value is greater than the first preset threshold, determine that the object in the first image to be detected belongs to an unknown category.

[0109] The second difference value is determined based on the calculated first difference value.

[0110] It is understandable that the second difference value can be determined based on the first difference value. If there are multiple first difference values, the second difference value can be determined based on the multiple first difference values. For example, the average value of the multiple first difference values ​​is used as the second difference value, or the largest first difference value among the multiple first difference values ​​is used as the second difference value. If there is one first difference value, the first difference value can be directly used as the second difference value. Specifically, the specific implementation method of how the second difference value is determined based on the first difference value will be described in detail in the subsequent embodiments.

[0111] Based on the above processing, the trained classification model can accurately identify images of real categories (that is, known categories). Since the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category, and each preset category includes the real category of the object in the sample image; thus, the first predicted probability distribution represents the predicted probability that the object in the second image to be detected belongs to each known category. Since the predicted probabilities of belonging to each preset category output by the classification model for images of unknown categories are close, the predicted probability distribution output by the classification model for images of unknown categories is relatively uniform. The corresponding probabilities of each preset category in the reference probability distribution are the same, and the predicted probability output by the classification model for images of unknown categories can be simulated by the reference probability distribution. If the first difference value representing the difference between the first predicted probability distribution and the reference probability distribution is smaller, it means that the closer the first predicted probability distribution is to the reference probability distribution, the higher the possibility that the object in the second image to be detected is an unknown category; further, if the second difference value determined based on the first difference value is not greater than the first preset threshold, it can be determined that the object in the first image to be detected belongs to an unknown category. It can be seen that the image detection method provided by the embodiment of the present invention can distinguish objects of known categories from objects of unknown categories to improve the accuracy of image detection.

[0112] Optionally, in one embodiment, as Figure 2 As shown, in Figure 1 Based on the image detection method shown, the image detection method further includes step S201.

[0113] S201, for each element in the first image feature, if the value of the element is greater than a second preset threshold, the value of the element is adjusted to the second preset threshold to obtain a second image feature.

[0114] The second preset threshold is determined based on elements in the image features obtained by extracting features from the sample image using a feature extraction network.

[0115] It is understandable that the feature extraction network in the trained classification model can accurately extract image features of images of known categories. However, for some difficult-to-identify images, such as images of unknown categories, the elements in the extracted image features will have some outliers. The size of these outliers is usually larger than the elements in the image features of known categories, which does not conform to the performance of the elements in the normal image feature vector.

[0116] In order to correct the abnormal value of the element in the image feature, after obtaining the first image feature using the feature extraction network, the larger element in the first image feature can be processed. If the element value is greater than the second preset threshold, the element value can be adjusted to the second preset threshold; if the element value is not greater than the second preset threshold, the element value is not adjusted to obtain the second image feature. The value of each element in the second image feature does not exceed the second preset threshold.

[0117] Exemplarily, if the classification model is a Deit-B model, the feature extraction network in the Deit-B model can perform feature extraction on the input second image to be detected, and obtain a feature vector h(x) of the first image feature with a dimension of m, where x represents the second image to be detected. m can be 768. The process of correcting the first image feature by the second preset threshold can be implemented by a Rectified Activations (ReAct) function. The ReAct function can truncate each element in the first image feature h(x). The specific truncation process is to correct the elements with higher values ​​in the first image feature to smaller values ​​by the second preset threshold. The ReAct function includes: ReAct(h(x);d)=min(x,d), where represents the second image feature, d represents the second preset threshold, and min(x,d) represents taking the smaller of the element in the first image feature and the second preset threshold. When the second preset threshold d tends to infinity, the processing of the ReAct function is equivalent to no correction, that is, the first image feature is almost the same as the second image feature.

[0118] In an embodiment of the present invention, a second preset threshold can be used to correct elements with higher values ​​in the first image features to smaller values, and the image features output by the feature extraction network can be post-processed and modified to reduce the noise caused by abnormal values, so that the second image features are closer to the ideal state, thereby improving the accuracy of the first predicted probability distribution obtained by subsequent classification using the second image features.

[0119] Optionally, in one implementation, a method for determining the second preset threshold includes steps A1-A3.

[0120] A1, select at least one sample image from all sample images used to train the classification model.

[0121] A2, inputting each sample image among the selected sample images into the feature extraction network of the trained classification model to obtain the third image feature of the sample image.

[0122] A3, determining a second preset threshold based on the obtained target value of each third image feature.

[0123] For each element in the third image feature, the proportion of elements greater than the target value in the third image feature to each element in the third image feature is a specified proportion.

[0124] It is understandable that the sample image used to train the classification model can be accurately identified by the trained classification model, and the feature extraction network in the trained classification model can be used to extract features from the sample image to obtain the third image feature of the sample image, and the second preset threshold is obtained according to the target value in the third image feature. Among them, among the elements of each third image feature, the proportion of elements greater than the target value in the elements of the third image feature is a specified proportion, for example, the specified proportion is 10%, and the third image feature contains elements 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and there is 1 element greater than 9, that is, 9 is the target value.

[0125] Exemplarily, the elements in the third image feature may be sorted in ascending order, and the elements may be arranged in ascending order. For each element in the third image feature, if the number of elements greater than the element accounts for a specified proportion of the total number of elements, the element is determined as the target value.

[0126] In the embodiment of the present invention, an accurate second preset threshold can be obtained by processing elements in the sample image, thereby improving the accuracy of the second image feature obtained by subsequently processing the first image feature through the second preset threshold.

[0127] Optionally, in one implementation, step A3 includes steps A31-A32.

[0128] A31, if there are multiple sample images selected, taking the average value of the sum of the target values ​​in the multiple third image features obtained as the second preset threshold;

[0129] A32: If only one sample image is selected, the target value in the third image feature of the sample image is obtained as the second preset threshold.

[0130] It is understandable that the target value can be determined using one or more sample images. If there are multiple sample images, the third image features of the multiple sample images can be obtained, and then the average value of the sum of the target values ​​in each third image feature can be used as the second preset threshold. If there is one sample image, the target value of the sample image can be directly used as the second preset threshold. In this implementation, the average value of multiple target values ​​can be used as the second preset threshold to improve the accuracy of the second preset threshold.

[0131] Optionally, in one embodiment, Figure 1Based on the image detection method shown, there are two situations: the second image to be detected is one, or the second image to be detected is multiple.

[0132] When there is one second image to be detected, the second image to be detected is the same as the first image to be detected, and the first difference value is the same as the second difference value.

[0133] It can be understood that when there is only one second image to be detected, the second image to be detected is the same as the first image to be detected, and the subsequent processing of the second image to be detected is the processing of the first image to be detected. The first difference value is the same as the second difference value. At this time, the first difference value can be directly compared with the first preset threshold to determine whether the object in the first image to be detected belongs to an unknown category.

[0134] When there are multiple second images to be detected, the second images to be detected include: multiple images obtained by performing specified image processing on the first image to be detected; the specified image processing includes at least one of the following: cropping, flipping. In this case, if Figure 3 As shown, the image detection method further includes step S301.

[0135] S301, calculating an average value of the sum of each first difference value obtained based on each second image to be detected, to obtain a second difference value.

[0136] It can be understood that when there are multiple second images to be detected, in the process of obtaining the second images to be detected based on the first image to be detected, the first image to be detected can be cropped and / or flipped to obtain multiple second images to be detected.

[0137] When there are multiple second images to be detected, use the feature extraction network to extract features from each second image to be detected to obtain multiple first image features; obtain a second image feature based on each first image feature, that is, based on multiple first image features, multiple second image features can be obtained; use the linear classification layer to classify each second image feature to obtain a first predicted probability distribution, and process multiple second image features to obtain multiple first predicted probability distributions. Calculate the first difference value of the difference between each first predicted probability distribution and the reference probability distribution, and multiple first difference values ​​can be obtained for multiple first predicted probability distributions. Calculate the average of the sum of each first difference value obtained based on each second image to be detected to obtain a second difference value.

[0138] Exemplarily, the test-time augmentation (TTA) process can be performed by using the ten-time random cropping average (Ten Crop Average) method. Specifically, the object in the first image to be detected can be used as the center to crop out the five regions of the center top, center bottom, center left, center right and center of the first image to be detected, and the cropped regions include the object. The five cropped regions are horizontally flipped to obtain ten second images to be detected. The ten second images to be detected are processed according to the above embodiment to obtain ten first difference values. The ten first difference values ​​can be averaged according to the formula for average processing to obtain the second difference value. Among them, the formula for average processing is: Among them, S(x g ) represents the g-th first difference value, the maximum value of g is 10, and S(x) represents the second difference value.

[0139] In the embodiment of the present invention, the test results can be effectively smoothed by taking the average value of multiple first difference values, thereby reducing output fluctuations caused by changes in input data, improving the robustness and generalization ability of the first difference values, and reducing the occurrence of misjudgment of data of unknown categories.

[0140] In one embodiment, if Figure 4 As shown, in Figure 3 On the basis of the image detection method shown, when the second image to be detected includes multiple images, the method further includes steps S401-S402.

[0141] S401, if the second difference value is not greater than the first preset threshold, for each preset category, the average value of the sum of the prediction probabilities of the preset category in each first prediction probability distribution is used as the prediction probability of the preset category to obtain a second prediction probability distribution.

[0142] It can be understood that if the second difference value is not greater than the first preset threshold, it can be determined that the object in the first image to be detected is a known category. In order to further determine which known category the object in the first image to be detected belongs to, the first predicted probability distributions of multiple second images to be detected can be superimposed and averaged. That is, for each preset category, the average of the sum of the predicted probabilities of the preset category in each first predicted probability distribution is used as the predicted probability of the preset category, and the second predicted probability distribution of the first image to be detected is obtained. For example, the preset categories include categories A, B, C, and D. Based on the first image to be detected, two second images to be detected can be obtained. The first predicted probability distribution of one second image to be detected includes: the probability that the object belongs to category A is 97%, the probability that the object belongs to category B is 1%, the probability that the object belongs to category C is 1%, and the probability that the object belongs to category D is 1%; the first predicted probability distribution of another second image to be detected includes: the probability that the object belongs to category A is 90%, the probability that the object belongs to category B is 5%, the probability that the object belongs to category C is 3%, and the probability that the object belongs to category D is 2%; then, the second predicted probability distribution of the first image to be detected includes: the probability that the object belongs to category A is (97+90) / 2=93.5%, the probability that the object belongs to category B is (1+5) / 2=3%, the probability that the object belongs to category C is (1+3) / 2=2%, and the probability that the object belongs to category D is (1+2) / 2=1.5%.

[0143] S402: Taking the preset category with the largest prediction probability in the second prediction probability distribution as the category of the object in the first image to be detected.

[0144] It can be understood that the second predicted probability distribution includes: the predicted probability that the object in the first image to be detected belongs to each preset category, and the preset category with the largest predicted probability in the second predicted probability distribution can be used as the category of the object in the first image to be detected. For example, the second predicted probability distribution of the first image to be detected includes: the probability that the object belongs to category A is 93.5%, the probability that the object belongs to category B is 3%, the probability that the object belongs to category C is 2%, and the probability that the object belongs to category D is 1.5%. The probability of 93.5% is the largest, and it can be determined that the category of the object in the first image to be detected is category A.

[0145] Optionally, in another implementation, there is only one second image to be detected, and the second image to be detected is the first image to be detected. The first predicted probability distribution of the second image to be detected can be used as the second predicted probability distribution, and the preset category with the largest predicted probability in the second predicted probability distribution can be used as the category of the object in the first image to be detected.

[0146] In an embodiment of the present invention, when the second difference value is not greater than the first preset threshold, the possibility that the object in the first image to be detected is a position category can be ruled out, and the known category of the object in the first image to be detected can be determined, thereby avoiding the situation where a known category is misjudged as an unknown category, and improving the accuracy of image detection.

[0147] In order to more clearly understand the embodiment of the present invention, an introduction is given below in conjunction with a schematic diagram of the principle of the image detection method.

[0148] like Figure 5 As shown, the first image to be detected can be subjected to test enhancement processing, and multiple second images to be detected can be obtained based on the first image to be detected through the test enhancement processing. The feature extraction network is used to extract features of the second image to be detected to obtain the first image feature; the ReAct function is used to correct the elements with higher values ​​in the first image feature to a second preset threshold with smaller values ​​to obtain the second image feature. This process can be called correction processing. The second image feature is classified using a linear classification network to obtain a first predicted probability distribution. A first difference value is calculated to represent the difference between the first predicted probability distribution and the reference probability distribution. The process of calculating the first difference value can also be called gradient normalization (GradNorm) processing. The mean of the sum of the first difference values ​​of each second image to be detected is used as the second difference value. If the second difference value is not greater than the first preset threshold, it is determined that the object in the first image to be detected belongs to an unknown category.

[0149] In the embodiment of the present invention, after the classification model training is completed, the trained classification model can be used to process the input image to be detected, without the need for additional model training. In addition, the trained classification model can accurately identify images of real categories (that is, known categories), because the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category, and each preset category includes the real category of the object in the sample image; thus, the first predicted probability distribution represents the predicted probability that the object in the second image to be detected belongs to each known category. Since the predicted probabilities of belonging to each preset category output by the classification model for images of unknown categories are close, the predicted probability distribution output by the classification model for images of unknown categories is relatively uniform. The corresponding probabilities of each preset category in the reference probability distribution are the same, and the predicted probability output by the classification model for images of unknown categories can be simulated by the reference probability distribution. If the first difference value representing the difference between the first predicted probability distribution and the reference probability distribution is smaller, it means that the first predicted probability distribution is closer to the reference probability distribution, and the possibility that the object in the second image to be detected is an unknown category is higher; further, if the second difference value determined based on the first difference value is not greater than the first preset threshold, it can be determined that the object in the first image to be detected belongs to an unknown category. It can be seen that the image detection method provided by the embodiment of the present invention can distinguish objects of known categories from objects of unknown categories, so as to improve the accuracy of image detection.

[0150] It should be noted that there is an existing solution 1 in the prior art: Maximum Softmax Probability (MSP), if the maximum probability of the preset category output by the classification model does not exceed the threshold, it is determined that the object of the input image is an unknown category; existing solution 2: Energy-Based OOD Detection (EBO), a large number of unknown category images are used to train the classification model to improve the classification model's recognition ability for unknown category images. When using an existing image set (for example, the image network ImageNet-1k & 21k) for testing, the existing solution 1, the existing solution 2, the solutions provided by steps s101-S105, S1041-S1042 and S201 of the embodiment of the present invention are used respectively to test the existing image set to obtain the results shown in the indicator comparison table in Table 1 below. Among them, the test indicators used in the indicator comparison table include: Area Under the Receiver Operating Characteristic Curve (AUROC), False Positive Rate at 95% True Positive Rate (FPR@TPR-95%), and Accuracy (ACC). Among them, the larger the AUROC and ACC, the smaller the FPR@TPR-95%, indicating that the accuracy of the scheme detection is higher.

[0151] AUROC FPR@TPR - 95% ACC Existing solution 1 76.09 73.02 83.75 Existing solution 2 77.16 70.01 83.75 The solution provided by the present invention 80.86 63.22 84.29

[0152] Table 1

[0153] As shown in Table 1, the AUROC and ACC of the solution provided by the present invention are greater than those of the existing solutions 1 and 2, and the FPR@TPR-95% is less than that of the existing solutions 1 and 2. That is to say, the solution of the present invention has a lower false alarm rate and a higher accuracy rate than the existing solutions 1 and 2. The solution of the present invention is carried out after the classification model is trained, which can avoid training the classification model and reduce the amount of training. It is suitable for various classification tasks. For some special scenarios, such as medical, autonomous driving, intelligent security and other fields, it can effectively reduce false alarms and improve detection accuracy.

[0154] Optionally, in one embodiment, as Figure 6 As shown, an embodiment of the present invention further provides an image detection device, the device comprising:

[0155] A first acquisition module 610 is used to acquire a first image to be detected and a trained classification model; wherein the acquired classification model is obtained by training using each sample image and a label representing a true category of an object in the sample image; the classification model includes a feature extraction network and a linear classification network;

[0156] The first utilizing module 620 is used to utilize a feature extraction network to perform feature extraction on the second image to be detected to obtain first image features; wherein the second image to be detected is obtained based on the first image to be detected;

[0157] The second utilization module 630 is used to classify the second image feature using a linear classification network to obtain a first prediction probability distribution; wherein the first prediction probability distribution includes: the prediction probability that the object in the second image to be detected belongs to each preset category; each preset category includes the real category of the object in the sample image; the second image feature is obtained based on the first image feature;

[0158] A first calculation module 640 is used to calculate a first difference value used to represent the difference between the first predicted probability distribution and the reference probability distribution; wherein the probabilities corresponding to the preset categories in the reference probability distribution are the same;

[0159] The first determination module 650 is used to determine that the object in the first to-be-detected image belongs to an unknown category if the second difference value is not greater than a first preset threshold value; wherein the second difference value is determined based on the calculated first difference value.

[0160] Optionally, the device further comprises:

[0161] An adjustment module is used to adjust the value of each element in the first image feature to the second preset threshold if the value of the element is greater than a second preset threshold, so as to obtain a second image feature; wherein the second preset threshold is determined based on the elements in the image feature obtained by extracting features from the sample image using a feature extraction network.

[0162] Optionally, a method for determining the second preset threshold includes:

[0163] Select at least one sample image from all sample images used to train the classification model;

[0164] Inputting each sample image among the selected sample images into the feature extraction network of the trained classification model to obtain a third image feature of the sample image;

[0165] Based on the target value obtained in each third image feature, a second preset threshold is determined; wherein, for the elements in each third image feature, the proportion of elements greater than the target value in the third image feature to each element in the third image feature is a specified proportion.

[0166] Optionally, determining a second preset threshold based on the obtained target value of each third image feature includes:

[0167] If there are multiple sample images selected, the average value of the sum of the target values ​​in the multiple third image features is taken as the second preset threshold;

[0168] If only one sample image is selected, the target value in the third image feature of the sample image is obtained as the second preset threshold.

[0169] Optionally, the second image to be detected is the same as the first image to be detected, and the first difference value is the same as the second difference value;

[0170] or,

[0171] The second image to be detected includes: multiple images obtained by performing specified image processing on the first image to be detected; the specified image processing includes at least one of the following: cropping, flipping; the device also includes: a second calculation module, used to calculate the average value of the sum of each first difference value obtained based on each second image to be detected, to obtain a second difference value.

[0172] The first calculation module 640 includes:

[0173] A utilizing unit, configured to calculate a first vector norm of the second image feature and a second vector norm of a difference between a predicted probability distribution and a reference probability distribution of the second image to be detected by utilizing a Manhattan norm function;

[0174] A determination unit is used to calculate the first vector norm, the second vector norm, the number of dimensions of the second image feature, and the total number of preset categories according to a specified formula to determine a first difference value; wherein the specified formula is: S represents the first difference value, U represents the first vector norm, V represents the second vector norm, feat_dim represents the number of dimensions of the second image feature, and num_classes represents the total number of preset categories.

[0175] Optionally, when the second image to be detected includes multiple images, the device further includes:

[0176] A second acquisition module is used for, if the second difference value is greater than the first preset threshold, taking, for each preset category, an average value of the sum of the prediction probabilities of the preset category in each first prediction probability distribution as the prediction probability of the preset category, to obtain a second prediction probability distribution;

[0177] The second determination module is used to use the preset category with the largest predicted probability in the second predicted probability distribution as the category of the object in the first image to be detected.

[0178] Based on the above processing, the trained classification model can accurately identify images of real categories (that is, known categories). Since the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category, and each preset category includes the real category of the object in the sample image; thus, the first predicted probability distribution represents the predicted probability that the object in the second image to be detected belongs to each known category. Since the predicted probabilities of belonging to each preset category output by the classification model for images of unknown categories are close, the predicted probability distribution output by the classification model for images of unknown categories is relatively uniform. The corresponding probabilities of each preset category in the reference probability distribution are the same, and the predicted probability output by the classification model for images of unknown categories can be simulated by the reference probability distribution. If the first difference value representing the difference between the first predicted probability distribution and the reference probability distribution is smaller, it means that the closer the first predicted probability distribution is to the reference probability distribution, the higher the possibility that the object in the second image to be detected is an unknown category; further, if the second difference value determined based on the first difference value is not greater than the first preset threshold, it can be determined that the object in the first image to be detected belongs to an unknown category. It can be seen that the image detection method provided by the embodiment of the present invention can distinguish objects of known categories from objects of unknown categories to improve the accuracy of image detection.

[0179] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701 , a communication interface 702 , a memory 703 and a communication bus 704 , wherein the processor 701 , the communication interface 702 , and the memory 703 communicate with each other via the communication bus 704 .

[0180] Memory 703, used for storing computer programs;

[0181] The processor 701 is used to implement the image detection method provided in the above embodiment when executing the program stored in the memory 703.

[0182] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0183] The communication interface is used for communication between the above electronic device and other devices.

[0184] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0185] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0186] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image detection methods are implemented.

[0187] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any image detection method in the above embodiments.

[0188] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0189] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0190] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0191] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An image detection method, characterized in that: The method comprises: Acquire a first image to be detected and a trained classification model; wherein the acquired classification model is obtained by training using each sample image and a label representing the true category of the object in the sample image; the classification model includes a feature extraction network and a linear classification network; Using the feature extraction network to extract features from a second image to be detected, to obtain features of the first image; wherein the second image to be detected is obtained based on the first image to be detected; The second image feature is classified by using the linear classification network to obtain a first prediction probability distribution; wherein the first prediction probability distribution includes: the prediction probability that the object in the second to-be-detected image belongs to each preset category; each preset category includes the real category of the object in the sample image; the second image feature is obtained based on the first image feature; Calculating a first difference value for indicating a difference between the first predicted probability distribution and a reference probability distribution; wherein the probabilities corresponding to the preset categories in the reference probability distribution are the same; If the second difference value is not greater than the first preset threshold, it is determined that the object in the first to-be-detected image belongs to an unknown category; wherein the second difference value is determined based on the calculated first difference value.

2. The method according to claim 1, characterized in that Before classifying the second image feature using the linear classification network to obtain the first predicted probability distribution, the method further includes: For each element in the first image feature, if the value of the element is greater than a second preset threshold, the value of the element is adjusted to the second preset threshold to obtain a second image feature; wherein the second preset threshold is determined based on the elements in the image feature obtained by extracting features from the sample image using the feature extraction network.

3. The method according to claim 2, characterized in that The second preset threshold is determined by: Selecting at least one sample image from all sample images used to train the classification model; Inputting each sample image among the selected sample images into the feature extraction network of the trained classification model to obtain a third image feature of the sample image; Based on the target value obtained in each third image feature, a second preset threshold is determined; wherein, for the elements in each third image feature, the proportion of elements greater than the target value in the third image feature to each element in the third image feature is a specified proportion.

4. The method according to claim 3, characterized in that The step of determining a second preset threshold based on the obtained target value of each third image feature comprises: If there are multiple sample images selected, the average value of the sum of the target values ​​in the multiple third image features is taken as the second preset threshold; If only one sample image is selected, the target value in the third image feature of the sample image is obtained as the second preset threshold.

5. The method according to claim 1, characterized in that The second image to be detected is the same as the first image to be detected, and the first difference value is the same as the second difference value; or, The second image to be detected includes: multiple images obtained by performing specified image processing on the first image to be detected; the specified image processing includes at least one of the following: cropping, flipping; before determining that the object in the first image to be detected belongs to an unknown category if the second difference value is greater than the first preset threshold, it also includes: calculating the average value of the sum of each first difference value obtained based on each second image to be detected to obtain a second difference value.

6. The method according to claim 1, characterized in that The calculating a first difference value representing a difference between the first predicted probability distribution and a reference probability distribution includes: Calculating, by using a Manhattan norm function, a first vector norm of the second image feature and a second vector norm of a difference between a predicted probability distribution and a reference probability distribution of the second image to be detected; The first vector norm, the second vector norm, the number of dimensions of the second image feature, and the total number of the preset categories are calculated according to a specified formula to determine a first difference value; wherein the specified formula is: S represents the first difference value, U represents the first vector norm, V represents the second vector norm, feat_dim represents the number of dimensions of the second image feature, and num_classes represents the total number of the preset categories.

7. The method according to claim 5, characterized in that When the second image to be detected includes a plurality of images, the method further includes: If the second difference value is greater than the first preset threshold, for each preset category, taking the average value of the sum of the predicted probabilities of the preset category in each of the first predicted probability distributions as the predicted probability of the preset category, to obtain a second predicted probability distribution; The preset category with the largest predicted probability in the second predicted probability distribution is used as the category of the object in the first image to be detected.

8. An image detection device, characterized in that: The device comprises: A first acquisition module is used to acquire a first image to be detected and a trained classification model; wherein the acquired classification model is obtained by training using each sample image and a label representing the true category of the object in the sample image; the classification model includes a feature extraction network and a linear classification network; A first utilizing module is used to utilize the feature extraction network to perform feature extraction on a second image to be detected to obtain first image features; wherein the second image to be detected is obtained based on the first image to be detected; A second utilization module is used to classify the second image feature using the linear classification network to obtain a first predicted probability distribution; wherein the first predicted probability distribution includes: the predicted probability that the object in the second image to be detected belongs to each preset category; each preset category includes the real category of the object in the sample image; the second image feature is obtained based on the first image feature; A first calculation module, used to calculate a first difference value used to represent the difference between the first predicted probability distribution and a reference probability distribution; wherein the probabilities corresponding to the preset categories in the reference probability distribution are the same; The first determination module is used to determine that the object in the first to-be-detected image belongs to an unknown category if the second difference value is not greater than a first preset threshold; wherein the second difference value is determined based on the calculated first difference value.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images