An image detection method and device, electronic equipment and storage medium

Abstract

The application discloses an image detection method and device, electronic equipment and storage medium. The method comprises the following steps: inputting an image to be detected into a feature extraction network of an image detection model for feature extraction, so as to obtain image features of the image to be detected; inputting the image features into a full connection network of the image detection model for feature mapping, so as to obtain a feature mapping result; the feature mapping result comprises response values of the image features to each preset category in a plurality of preset categories; performing energy calculation on the response values of the image features to each preset category in the plurality of preset categories based on an energy function, so as to obtain an uncertainty of the image to be detected; the uncertainty represents an uncertainty degree of the image to be detected belonging to the plurality of preset categories; when the uncertainty exceeds a preset uncertainty threshold, outputting a first detection result; the first detection result indicates that the image to be detected is an abnormal input image. The application improves the detection accuracy of the abnormal input image of the category not involved in the training data.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image detection method, apparatus, electronic device and storage medium. Background Technology

[0002] With the development of computer technology, artificial intelligence (AI) technology based on computer vision has been widely used in various industries. For example, AI industrial defect inspection uses AI technology based on computer vision, such as image detection models based on deep learning, to inspect the quality of industrial products in the manufacturing process, in order to detect whether there are defects and the type of defects in the input industrial product images.

[0003] In related technologies, image detection models are trained on training samples of known categories. During image detection, the highest confidence score output by the model is used to determine whether the current input is an anomalous image (i.e., an image not included in the training samples). Typically, an input image is considered anomalous when the highest confidence score is relatively low, and an anomalous detection result is output. However, in practical applications, if the image detection model is overfitted, it may output a high confidence score even when the input is an anomalous image. This leads to inaccurate detection results for anomalous input images in related technologies, making it impossible to process anomalous inputs in a timely manner. Summary of the Invention

[0004] To address the problems of existing technologies, this application provides an image detection method, apparatus, electronic device, and storage medium. The technical solution is as follows:

[0005] On the one hand, an image detection method is provided, the method comprising:

[0006] The image to be detected is acquired, and the image to be detected is input into the feature extraction network of the image detection model to extract features, thereby obtaining the image features corresponding to the image to be detected.

[0007] The image features are input into the fully connected network of the image detection model for feature mapping to obtain the feature mapping result; the feature mapping result includes the response value of the image features to each of the multiple preset categories;

[0008] Using an energy function, energy is calculated for the response value of each of the multiple preset categories based on the image features to obtain the uncertainty of the image to be detected; the uncertainty characterizes the degree of uncertainty of the image to be detected belonging to the multiple preset categories;

[0009] When the uncertainty exceeds a preset uncertainty threshold, a first detection result is output; the first detection result indicates that the image to be detected is an abnormal input image.

[0010] On the other hand, an image detection apparatus is provided, the apparatus comprising:

[0011] The image feature extraction module is used to acquire the image to be detected, input the image to be detected into the feature extraction network of the image detection model for feature extraction, and obtain the image features corresponding to the image to be detected.

[0012] The feature mapping module is used to input the image features into the fully connected network of the image detection model for feature mapping to obtain the feature mapping result; the feature mapping result includes the response value of the image features to each of the multiple preset categories;

[0013] An uncertainty calculation module is used to calculate the energy of the response value of each of the multiple preset categories based on the image features using an energy function, thereby obtaining the uncertainty of the image to be detected; the uncertainty characterizes the degree of uncertainty of the image to be detected belonging to the multiple preset categories;

[0014] The first detection result output module is used to output a first detection result when the uncertainty exceeds a preset uncertainty threshold; the first detection result indicates that the image to be detected is an abnormal input image.

[0015] In one exemplary embodiment, the apparatus further includes:

[0016] A normalization processing module is used to normalize the feature mapping result when the uncertainty does not exceed the preset uncertainty threshold, to obtain a normalized result; the normalized result represents the probability that the image to be detected belongs to each of the plurality of preset categories;

[0017] The target category determination module is used to select the preset category with the highest probability based on the normalization result, as the target category of the image to be detected;

[0018] The second detection result output module is used to output a second detection result based on the target category of the image to be detected; the second detection result indicates the probability that the image to be detected belongs to the target category.

[0019] In one exemplary embodiment, the apparatus further includes a training module for training the image detection model, the training module comprising:

[0020] The first acquisition module is used to acquire the image detection model to be trained and the sample image set; the image detection model to be trained is obtained by training a preset neural network model on an image classification task.

[0021] The first sample feature extraction module is used to extract features from each sample image in the sample image set based on the feature extraction network of the image detection model to be trained, so as to obtain the sample image features of each sample image.

[0022] The probability density determination module is used to determine the probability density corresponding to each sample image feature based on the sample image features corresponding to each sample image of the preset category, and to sample target sample image features based on the probability density corresponding to each sample image feature; the probability density corresponding to the target sample image feature is less than a preset probability density threshold.

[0023] The sample feature mapping module is used to perform feature mapping on each sample image feature based on the fully connected network of the image detection model to be trained, so as to obtain the sample feature mapping result corresponding to each sample image feature; the sample feature mapping result includes the response value of the corresponding sample image feature to each of the multiple preset categories;

[0024] The sample uncertainty calculation module is used to calculate the uncertainty of the response value of each of the plurality of preset categories based on each of the sample image features, so as to obtain the prediction uncertainty corresponding to each of the sample image features;

[0025] The training submodule is used to train the image detection model to be trained with the goal of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty, until the preset training termination condition is met to obtain the image detection model; wherein, the first comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each of the sample image features, and the second comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each of the target sample image features.

[0026] In one exemplary embodiment, the probability density determination module includes:

[0027] The first determining module is configured to, for each preset category, determine the mean value of the sample image features of the preset category to obtain the feature mean of the preset category; and determine the feature covariance of the preset category based on the difference between the sample image features of each preset category and the feature mean of the preset category.

[0028] The second determining module is used to obtain a covariance matrix based on the sum of the feature covariances of each of the plurality of preset categories;

[0029] The probability density determination submodule is used to determine the probability density corresponding to each sample image feature based on the covariance matrix and the difference between the sample image feature and the feature mean of the preset category corresponding to the sample image feature.

[0030] In one exemplary implementation, each sample image in the sample image set corresponds to label information, the label information indicating the preset category of the corresponding sample image among the plurality of preset categories; the training submodule includes:

[0031] An uncertainty loss construction module is used to construct uncertainty loss with the objective of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty;

[0032] The prediction and classification module is used to normalize the sample feature mapping results corresponding to the features of each sample image to obtain the prediction and classification results for each sample image; the prediction and classification results represent the probability that the corresponding sample image belongs to each of the preset categories;

[0033] The classification loss determination module is used to determine the classification loss based on the predicted classification result corresponding to each sample image and the label information of each sample image;

[0034] The comprehensive loss determination module is used to perform a weighted summation of the uncertain loss and the classification loss to obtain the comprehensive loss;

[0035] The parameter adjustment module is used to adjust the model parameters of the image detection model to be trained based on the comprehensive loss until the preset training termination condition is met to obtain the image detection model.

[0036] In one exemplary embodiment, the apparatus further includes:

[0037] The second acquisition module is used to acquire the training image set;

[0038] The image detection model determination module is used to train the preset neural network model on the training image set to obtain the image detection model to be trained; the image classification task is to predict the probability that the training images in the training image set belong to each of the preset categories.

[0039] In one exemplary implementation, the uncertainty calculation is based on an energy function.

[0040] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image detection method of any of the above aspects.

[0041] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the image detection method as described above.

[0042] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the image detection method of any of the above aspects.

[0043] This application embodiment extracts image features of the image to be detected through a feature extraction network based on an image detection model, and inputs these image features into a fully connected network of the image detection model for feature mapping to obtain response values ​​of the image features to each preset category. Then, an energy function is used to calculate the energy based on the response values ​​of the image features to each preset category to obtain the uncertainty corresponding to the image to be detected. When the uncertainty exceeds a preset uncertainty threshold, a first detection result indicating that the image to be detected is an abnormal input image is output, thereby greatly improving the detection accuracy of abnormal input images that are not involved in the training data. When applied to AI industrial defect quality inspection scenarios, it can promptly alarm when abnormal input images are detected to avoid batch losses and accidents during the production process, which is conducive to improving the reliability of AI industrial defect quality inspection. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of existing technology for detecting abnormal input images;

[0046] Figure 2 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0047] Figure 3This is a schematic flowchart of an image detection method provided in an embodiment of this application;

[0048] Figure 4 This is a schematic flowchart of the training image detection model provided in the embodiments of this application;

[0049] Figure 5 This is another flowchart illustrating the training image detection model provided in this application embodiment;

[0050] Figure 6 This is an example provided in this application of sampling target sample image features from a low-density region in the hidden feature space of the image detection model to be trained, and training the image detection model to be trained;

[0051] Figure 7 This is another flowchart illustrating the training image detection model provided in this application embodiment;

[0052] Figure 8 This is a structural block diagram of an image detection device provided in an embodiment of this application;

[0053] Figure 9 This is a hardware structure block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0056] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0057] In related technologies, image detection models are trained based on training samples of known categories. During the image detection process, the highest confidence score output by the image detection model is used to determine whether the current input is an abnormal input image (i.e., an image that does not involve any category in the training samples). Usually, when the highest confidence score output is relatively low, the input image is considered to be an abnormal input image, and an abnormal detection result will be output.

[0058] Taking AI-powered industrial defect inspection as an example, in actual production, some extremely rare defect types often occur, or the AI ​​itself may encounter input anomalies (such as damaged input images due to imaging reasons, i.e., black images, out-of-focus images, etc.). This poses a challenge to the already deployed image detection models, because the training phase of image detection models is based on images of known defect categories. Therefore, for unknown categories (i.e., abnormal inputs), unreliable prediction results will be output. However, in real-world scenarios, these abnormal inputs need to be captured and alerted by the AI ​​system in a timely manner to avoid safety accidents or economic losses.

[0059] like Figure 1 The diagram illustrates an existing technique for detecting anomalous input images. Specifically, the process for determining anomalous input images involves first obtaining the highest confidence level (probability p) output by the model. max Then, based on the highest confidence level p max Based on ∈[0,1], the highest confidence level p max The higher (e.g.) Figure 1 A value greater than 0.8 indicates lower uncertainty, suggesting a high probability of a normal input image (i.e., belonging to a category known to the model). Therefore, the output reflects the corresponding model-detected category (e.g., ...). Figure 1 The results above indicate that the defect category is defect 1; conversely, if the highest confidence level p max Relatively low (e.g.) Figure 1 If the uncertainty is less than 0.8, then the uncertainty is high, which means that the input image is likely an abnormal input image (that is, an image that is not involved in the training samples and does not belong to the category known by the model). At this time, the abnormal judgment result will be output and finally fed back to the business personnel to remind the model that an abnormal input has been detected and human intervention is required for confirmation and processing.

[0060] It is evident that the aforementioned methods for detecting abnormal input images in related technologies heavily rely on the highest confidence output of the image detection model. However, in practical applications, when the image detection model is overfitted, it will still output a high confidence level even when the input image is abnormal. This results in inaccurate detection results for abnormal input images in related technologies, posing a high risk of erroneous judgments and hindering timely processing of abnormal inputs.

[0061] In view of this, this application provides an image detection method. It extracts image features of the image to be detected through a feature extraction network based on an image detection model, and inputs these image features into a fully connected network of the image detection model for feature mapping to obtain response values ​​for each preset category. Then, it uses an energy function to calculate the energy based on the response values ​​of the image features to each preset category to obtain the uncertainty corresponding to the image to be detected. When the uncertainty exceeds a preset uncertainty threshold, it outputs a first detection result indicating that the image to be detected is an abnormal input image. This greatly improves the detection accuracy for abnormal input images that are not involved in the training samples. When applied to AI industrial defect inspection scenarios, it can promptly issue an alarm when an abnormal input image is detected to avoid batch losses and accidents during production, thus improving the reliability of AI industrial defect inspection.

[0062] Please see Figure 2 The diagram shown is an implementation environment provided in an embodiment of this application. The implementation environment includes a terminal 210 and a server 220, wherein the terminal 210 and the server 220 can communicate via a wired or wireless network connection.

[0063] Terminal 210 includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. Terminal 210 is equipped with client software, such as an application (App), that has image detection capabilities. This application can be a standalone application or a subroutine within an application. The image detection function can detect whether the input image is an anomalous input image. If it is an anomalous input image, it outputs a detection result indicating that it is an anomalous input image; if it is not an anomalous input image (i.e., a normal input image), it outputs an indication of the detected preset category. An anomalous input image refers to an image whose category is not covered in the training samples.

[0064] Server 220 can provide background services for the application in terminal 210. Specifically, this background service can be an image detection service. In particular, server 220 can receive the image to be detected sent by terminal 210, call a pre-trained image detection model, perform detection on the image to be detected based on the image detection model, and return the detection result to terminal 210 so that terminal 210 can perform corresponding processing based on the received detection result. For example, when the detection result indicates that the image to be detected is an abnormal input image, terminal 210 can issue an alarm message to prompt the image detection model to detect an abnormal input image, so that business personnel can handle it in a timely manner.

[0065] It should be noted that the server involved in the embodiments of this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0066] In one exemplary embodiment, both terminal 210 and server 220 can be node devices in a blockchain system, capable of sharing acquired and generated information with other node devices in the blockchain system, thus enabling information sharing among multiple node devices. Multiple node devices in the blockchain system can be configured with the same blockchain, which consists of multiple blocks, and adjacent blocks are related, ensuring that any data tampering in any block can be detected by the next block. This prevents data tampering in the blockchain and guarantees the security and reliability of the data in the blockchain.

[0067] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0068] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0069] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0070] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0071] Please see Figure 3 The diagram illustrates a flowchart of an image detection method provided in this application. This method can be applied to an electronic device, which may be a terminal or a server. It should be noted that this specification provides the operational steps of the method as described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many steps and does not represent the only execution order. In actual systems or products, the methods can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or accompanying drawings. Specifically, as shown... Figure 3 As shown, the method may include:

[0072] S301, acquire the image to be detected, input the image to be detected into the feature extraction network of the image detection model to extract features, and obtain the image features corresponding to the image to be detected.

[0073] The image to be detected is the image that needs to be inspected. For example, in the AI ​​industrial defect inspection scenario, the image to be detected can be an image of an industrial product in the manufacturing process, an overall image of an industrial product, or a partial image of an industrial product.

[0074] The image detection model can be a pre-trained neural network model, such as a convolutional neural network (CNN). A CNN can include an input layer, convolutional / pooling layers (pooling layers are optional), and fully connected layers (FC). The output of a convolutional layer can be used as the input to a subsequent pooling layer or as the input to another convolutional layer for further convolution operations. A convolutional layer can contain multiple convolution operators, also called kernels. In image processing, a convolution operator acts as a filter to extract specific information from the input image matrix. Essentially, a convolution operator can be a weight matrix, which is usually predefined. During the convolution operation, the weight matrix typically processes the input image pixel by pixel (or two pixels by two pixels, depending on the stride) along the horizontal direction, thereby extracting specific features from the image. Typically, when a convolutional neural network includes multiple convolutional layers, the initial convolutional layers tend to extract more general features, which can also be called low-level features. The features extracted by later convolutional layers become more and more complex, such as high-level semantic features.

[0075] In this embodiment of the application, the feature extraction network of the image detection model is used to extract features from the input image to be detected to obtain image features. For example, the feature extraction network can be multiple stacked convolutional layers. The image features are obtained by extracting features from the image to be detected through the convolution operation of each convolutional layer. Specifically, the image features can be the output of the last convolutional layer / pooling layer of the image detection model.

[0076] If x represents the image to be detected, then the image features can be represented as h(x) = f(x; θ1), where θ1 is the parameter of the feature extraction network, which can be determined through training; f(·) represents the feature extraction operation of the feature extraction network.

[0077] S303 inputs image features into the fully connected network of the image detection model for feature mapping to obtain the feature mapping result.

[0078] The feature mapping result is the output of the fully connected layer, which includes the response values ​​of image features to each of the multiple preset categories. The response value for each preset category can be understood as the unnormalized numerical score predicted for each preset category. Because it is unnormalized, the response values ​​for each preset category may not fall within a fixed numerical range.

[0079] Among these, multiple preset categories are categories that the image detection model can predict, typically categories known during training, i.e., the categories labeled on the samples. Understandably, the preset categories involved in different application scenarios can be different. For example, in an AI-driven industrial defect inspection scenario, the preset category could be a defect category, and thus multiple preset categories could be defect category 1, defect category 2, and so on.

[0080] Specifically, the fully connected network, also known as the fully connected layer FC, is used to transform the image features output by the feature extraction network into a K-dimensional vector, i.e., the feature mapping result, where K is the number of multiple preset categories. This fully connected network can include multiple hidden layers, the parameters of which can be pre-trained using training data relevant to the image classification task. For example, the feature mapping result output by the fully connected network can be represented as FC(h(x); θ2), where θ2 represents the parameters of the fully connected network, which can be determined through training; and FC(·) represents the feature mapping processing of the fully connected network.

[0081] S305, using the energy function, based on the above image features, the energy of the response value of each preset category in multiple preset categories is calculated to obtain the uncertainty of the image to be detected.

[0082] The result of energy calculation is used as the uncertainty of the image to be detected, which represents the degree of uncertainty of whether the image to be detected belongs to multiple preset categories.

[0083] For example, the uncertainty can be measured using the energy function E(x) shown in the following formula (1):

[0084]

[0085] Where x represents the image to be detected; k represents the kth preset category; K represents the total number of preset categories; h(x) represents the image features; FC(h(x); θ2) represents the feature mapping result.

[0086] Specifically, the larger the value of the energy function E(x), the greater the uncertainty; conversely, the smaller the value of the energy function E(x), the smaller the uncertainty.

[0087] S307, determine whether the uncertainty of the image to be detected exceeds the preset uncertainty threshold.

[0088] The preset uncertainty threshold can be set based on actual needs. Generally, the smaller the preset uncertainty threshold, the greater the probability of detecting abnormal input images; conversely, the larger the preset uncertainty threshold, the smaller the probability of detecting abnormal input images.

[0089] Specifically, if the determination result is that the uncertainty of the image to be detected exceeds the preset uncertainty threshold, it indicates that the image to be detected is likely to belong to an unknown category (i.e., not one of the multiple preset categories), and the following step S309 can be executed. Conversely, if the determination result is that the uncertainty of the image to be detected does not exceed the preset uncertainty threshold, it indicates that the image to be detected is likely to belong to one of the multiple preset categories, that is, the image to be detected is likely to be a regular input image, and the following step 311 can be executed.

[0090] S309, outputs the first detection result indicating that the image to be detected is an abnormal input image.

[0091] S311, Normalize the above feature mapping results to obtain the normalized result.

[0092] The normalization result represents the probability that the image to be detected belongs to each of the multiple preset categories.

[0093] In practice, normalization can be achieved using the normalization exponential function, i.e., the softmax function.

[0094] S313. Based on the above normalization results, select the preset category corresponding to the highest probability as the target category of the image to be detected.

[0095] S315, based on the target category of the image to be detected, outputs the second detection result.

[0096] The second detection result indicates the probability that the image to be detected belongs to the target category.

[0097] The above implementation utilizes an energy function to calculate the energy of the response value of each preset category among multiple preset categories based on the image features output by the fully connected network in the image detection model. The energy value, as the result of the energy calculation, represents the uncertainty. The magnitude of the uncertainty is then used to determine whether the image to be detected is an abnormal input image, thereby improving the detection accuracy of abnormal input images. Furthermore, when the image to be detected is determined to be a non-abnormal input image, i.e., a normal input image, the probability of belonging to the target category is further determined, and the detection result is output in combination with the probability of the target category, thus ensuring the accuracy of the category output for normal input images.

[0098] The training process of the image detection model described in the embodiments of this application is described below, specifically as follows: Figure 4As shown, the training process of this image detection model includes:

[0099] S401, Obtain the image detection model to be trained and the sample image set.

[0100] The image detection model to be trained is obtained by training a preset neural network model on an image classification task. This preset neural network model can be an initialized convolutional neural network. In other words, to train the image detection model in this embodiment from the preset neural network model, two stages of training are required. The first training stage is used to train the preset neural network model on an image classification task to obtain the image detection model to be trained, and the second training stage is used to train the image detection model to obtain the final image detection model.

[0101] It should be noted that the training data used in the first training stage and the second training stage can be the same or different. In this embodiment, for ease of distinction, the training data used in the first training stage is referred to as the training image set, and the training data used in the second training stage is referred to as the sample image set. Each sample image in the sample image set corresponds to label information, which indicates the preset category of the corresponding sample image among multiple preset categories. It can be understood that each training image in the training image set also corresponds to annotation information, which indicates the preset category of the corresponding training image among the aforementioned multiple preset categories.

[0102] Training the pre-defined neural network model for image classification in the first training phase ensures that the latent space of the trained image detection model satisfies a class-related multivariate Gaussian distribution. This means that the latent state vectors obtained from feature extraction of sample images of the same class by the trained image detection model conform to a multivariate Gaussian distribution. Where, x ′ Represents a sample image; This represents the feature mean of the k-th preset category; Represents the covariance matrix; h(x) ′ ) represents the hidden state vector, which can also be called the sample image feature.

[0103] Based on this, in some possible implementations, such as Figure 5 As shown, prior to step S401 above, the method may further include:

[0104] S501, Obtain the training image set.

[0105] Understandably, each training image in the training image set has corresponding annotation information, which indicates the preset category to which the corresponding training image belongs among multiple preset categories.

[0106] S503, based on the training image set, the preset neural network model is trained for a classification task to obtain the image detection model to be trained.

[0107] The classification task involves predicting the probability that a training image in the training image set belongs to each preset category.

[0108] In practice, training images are input into a preset neural network model. The feature extraction network of the preset neural network model extracts features to obtain training image features. These training image features are then passed through the fully connected network of the preset neural network model. The output of the fully connected network is processed by a normalized exponential function softmax to obtain the preset classification result corresponding to the training image. This predicted classification result indicates the probability that the training image belongs to each preset category. Then, based on the difference between the predicted classification result corresponding to each training image and the corresponding annotation information, a loss value is determined. Based on this loss value, the model parameters of the preset neural network model (including the parameters of the feature extraction network and the fully connected network) are adjusted in reverse. The model is then iteratively trained based on the adjusted model parameters until the preset training termination condition is reached to obtain the image detection model to be trained.

[0109] The preset training termination condition can be that the number of iterations reaches a preset number of iterations, the loss value reaches a preset loss threshold, or the difference between the loss values ​​of two adjacent iterations reaches a preset difference threshold.

[0110] The calculation of the above loss value can be based on the cross-entropy loss function, or other loss function forms, such as KL divergence, can be used.

[0111] Taking the AI ​​industrial defect inspection scenario as an example, the training image set can be images of industrial products, and the annotation information can be the defect category of industrial products. Then the loss value of the first stage of training can be expressed as the following formula (2):

[0112] L cls =CE(p(x″), y) (2)

[0113] Where CE(·) represents the cross-entropy loss function; x″ represents the training image; y represents the defect category of the industrial product image; p(x″) represents the predicted classification result, which can be expressed as p(x″) = softmax(FC(h(x″);θ2)) = {p1,p2,...,p K} represents the predicted probability values ​​corresponding to the K categories of defects.

[0114] S403, the feature extraction network based on the image detection model to be trained performs feature extraction on each sample image in the sample image set to obtain the sample image features of each sample image.

[0115] As mentioned above, the sample image features corresponding to sample images of the same preset category in the sample image set conform to a multivariate Gaussian distribution.

[0116] S405, based on the sample image features corresponding to the sample images of each preset category, determine the probability density corresponding to each sample image feature, and sample the target sample image features based on the probability density corresponding to each sample image feature.

[0117] Specifically, the probability density corresponding to the features of the target sample image is less than a preset probability density threshold. This preset probability density threshold can be understood as a hyperparameter, representing a very small neighborhood used to control the sampled target sample image features from deviating too much from the feature mean.

[0118] Understandably, the features of the target sample image are low-density regions in the hidden feature space (i.e., extracted image features) of the image detection model to be trained. Features in low-density regions of the hidden state space are usually close to the boundary of the model's decision, meaning they are hard sample image features for the model and can be considered as extracted features from abnormal input images. For example... Figure 5 The image shown illustrates an example of sampling target sample image features from low-density regions in the hidden feature space of the image detection model to be trained, and then using this sample image to train the model. Figure 6 The hidden feature space shown includes features located in high-density regions (i.e., within the distribution). Figure 6 Features within the middle circle and features located in low-density areas (outside the distribution) Figure 6 Features outside the middle circle).

[0119] Specifically, the probability density corresponding to each sample image feature can be a multivariate Gaussian distribution probability density. Based on this, the above determination of the probability density corresponding to each sample image feature based on the sample image features corresponding to each preset category can include:

[0120] For each preset category, the mean value of the sample image features of the preset category is determined to obtain the feature mean of the preset category, and the feature covariance of the preset category is determined based on the difference between the sample image features of the preset category and the feature mean of the preset category.

[0121] The covariance matrix is ​​obtained by summing the feature covariances of each of the multiple preset categories.

[0122] For each sample image feature, the probability density corresponding to the sample image feature is determined based on the covariance matrix mentioned above and the difference between the sample image feature and the feature mean of the preset category corresponding to the sample image feature.

[0123] In specific implementation, the covariance matrix and the feature mean of each preset category can be calculated using the sample expectation approximation method, as shown in the following formulas (3) and (4):

[0124]

[0125]

[0126] in, This represents the feature mean of the preset category k; The covariance matrix is ​​represented by N; N represents the total number of sample images in the sample image set; x i ′∈{x1′,x2′,...,x N '} represents sample image i in the sample image set; h(x) i ′) represents the sample image features of sample image i; y i =k indicates that the preset category of sample image i is k; This is an indicator function, indicating that only sample images of the preset category k will participate in the corresponding calculation process. Wherein, Represents the feature covariance of the preset category k; (·) T Indicates transpose; This indicates the number of sample images in the preset category k of the sample image set.

[0127] Then, for a sample image feature h in the preset category k k The probability density can be expressed as the following formula (5):

[0128]

[0129] Where m represents the sample image feature h k The dimension; (·) T This indicates transpose.

[0130] Therefore, the target sample image features sampled based on the probability density corresponding to each sample image feature can be expressed as the following formula (6):

[0131]

[0132] Among them, V k v represents the image features of all target samples of the preset category k. k ;∈ represents the preset probability density threshold;V represents the image features of all target samples.

[0133] S407, based on the fully connected network of the image detection model to be trained, performs feature mapping on each sample image feature to obtain the sample feature mapping result corresponding to each sample image feature.

[0134] The sample feature mapping result includes the response value of the corresponding sample image features to each of the multiple preset categories.

[0135] S409, using the energy function, calculate the energy of the response value of each preset category in multiple preset categories based on each sample image feature, and obtain the prediction uncertainty corresponding to each sample image feature.

[0136] The energy calculation result corresponding to each sample image feature is used as the prediction uncertainty for that sample image feature. The prediction uncertainty characterizes the degree of uncertainty regarding whether the sample image corresponding to a given sample image feature belongs to multiple preset categories.

[0137] For example, the measurement of prediction uncertainty can be done using the energy function E(x) shown in the aforementioned formula (1). Specifically, the larger the value of the energy function E(x), the greater the prediction uncertainty; conversely, the smaller the value of the energy function E(x), the smaller the prediction uncertainty.

[0138] Understandably, after the above step S409, the prediction uncertainty corresponding to each sample image feature can be obtained. In other words, the prediction uncertainty of each target sample image feature sampled from the low-density region is also obtained at this time.

[0139] S411, with the goal of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty, the image detection model to be trained is trained until the preset training termination condition is met to obtain the image detection model.

[0140] The first comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to the features of each sample image, and the second comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to the features of each target sample image.

[0141] Specifically, in this embodiment, the sum of the prediction uncertainties corresponding to the sample image features of all sample images in the sample image set is used to characterize the uncertainty within the entire distribution. That is, all sample images in the sample image set are regarded as regular input images, and the sum of the prediction uncertainties corresponding to the features of each sampled target sample image is used to characterize the uncertainty outside the entire distribution. That is, the features of each sampled target sample image are regarded as features corresponding to abnormal input images, i.e., hard sample image features, to increase the training difficulty of the image detection model to be trained. By minimizing the uncertainty within the distribution and maximizing the uncertainty outside the distribution, the decision boundary of the image detection model to be trained is reshaped to enhance the predictive robustness of the model uncertainty (i.e., to have a stronger classification ability for abnormal input images).

[0142] In practical applications, the uncertainty loss can be constructed with the objective of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty. For example, it can be constructed by calculating the expectation. The uncertainty loss is constructed in the form of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty, as shown in the following formula (7):

[0143]

[0144] Among them, L uncertainty denoted as uncertainty loss; V represents all target sample image features v obtained from sampling; D represents the sample image set, and x represents the sample image; E(v) represents the prediction uncertainty corresponding to the target sample image features characterized by the value of the energy function; E(x) represents the prediction uncertainty corresponding to the sample image features characterized by the value of the energy function.

[0145] In some possible implementations, such as Figure 6 as well as Figure 7 As shown, step S411 above may specifically include the following during implementation:

[0146] S701, with the objective of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty, an uncertainty loss is constructed.

[0147] The construction of the uncertain loss can be found in the above formula (7).

[0148] S703, normalize the sample feature mapping results corresponding to the features of each sample image to obtain the prediction classification result corresponding to each sample image.

[0149] The predicted classification result represents the probability that the corresponding sample image belongs to each of the preset categories.

[0150] The normalization process can be achieved using the normalization exponential function softmax.

[0151] S705, based on the predicted classification result corresponding to each sample image and the label information of each sample image, determines the classification loss.

[0152] Specifically, the classification loss is determined based on the difference between the predicted classification result corresponding to each sample image and the label information of that sample image. For example, the classification loss can be calculated based on the cross-entropy loss function. Of course, other loss function forms can also be used for the classification loss, such as KL divergence.

[0153] S707 calculates the weighted sum of the uncertain loss and the classified loss to obtain the comprehensive loss.

[0154] Specifically, the overall loss can be expressed as L = L cls +β Luncertainty , where L cls L represents the classification loss described above. uncertainty The above represents the uncertain loss. β is a hyperparameter used to control the proportion of the two losses, and it can usually be set based on actual needs.

[0155] S709, the model parameters of the image detection model to be trained are adjusted in reverse based on the comprehensive loss until the preset training termination condition is met to obtain the image detection model.

[0156] The preset training termination condition can be that the number of iterations reaches a preset number of iterations, the comprehensive loss reaches a preset loss threshold, or the difference between the comprehensive losses of two adjacent iterations reaches a preset difference threshold.

[0157] In this process, adjusting the model parameters of the image detection model to be trained involves adjusting the parameters of the feature extraction network and the fully connected network of the image detection model to be trained using backpropagation. The specific backpropagation method can be based on any applicable backpropagation method, such as batch gradient descent.

[0158] The above implementation method obtains high-quality abnormal input image samples by performing multivariate Gaussian fitting and low-density region sampling in the hidden feature space during the training phase of the image detection model. Since these abnormal samples are sampled in low-density regions with Gaussian distribution, they are relatively close to the decision boundary of the model, which increases the training difficulty of the model. They can be used as hard examples to reshape the decision boundary of the model, thereby improving the classification accuracy of the image detection model and enhancing the output robustness of the uncertainty of the image detection model. This makes the image detection model more accurate in predicting the uncertainty of abnormal input images, thus improving the accuracy of abnormal input image detection.

[0159] Corresponding to the image detection methods provided in the above embodiments, this application also provides an image detection device. Since the image detection device provided in this application corresponds to the image detection methods provided in the above embodiments, the implementation methods of the aforementioned image detection methods are also applicable to the image detection device provided in this embodiment, and will not be described in detail in this embodiment.

[0160] Please see Figure 8 The diagram shown is a structural schematic of an image detection device provided in an embodiment of this application. This device has the function of implementing the image detection method described in the above-described method embodiments. This function can be implemented in hardware or by hardware executing corresponding software. Figure 8 As shown, the image detection device 800 may include:

[0161] The image feature extraction module 810 is used to acquire the image to be detected, input the image to be detected into the feature extraction network of the image detection model for feature extraction, and obtain the image features corresponding to the image to be detected.

[0162] The feature mapping module 820 is used to input the image features into the fully connected network of the image detection model for feature mapping to obtain the feature mapping result; the feature mapping result includes the response value of the image features to each of the multiple preset categories;

[0163] The uncertainty calculation module 830 is used to calculate the energy of the response value of each of the multiple preset categories based on the image features using an energy function, thereby obtaining the uncertainty of the image to be detected; the uncertainty characterizes the degree of uncertainty of the image to be detected belonging to the multiple preset categories;

[0164] The first detection result output module 840 is used to output a first detection result when the uncertainty exceeds a preset uncertainty threshold; the first detection result indicates that the image to be detected is an abnormal input image.

[0165] In one exemplary embodiment, the device further includes:

[0166] The normalization processing module is used to normalize the feature mapping result when the uncertainty does not exceed the preset uncertainty threshold, and obtain the normalized result; the normalization result represents the probability that the image to be detected belongs to each of the multiple preset categories;

[0167] The target category determination module is used to select the preset category with the highest probability based on the normalization result, as the target category of the image to be detected.

[0168] The second detection result output module is used to output a second detection result based on the target category of the image to be detected; the second detection result indicates the probability that the image to be detected belongs to the target category.

[0169] In one exemplary embodiment, the apparatus further includes a training module for training the image detection model, the training module comprising:

[0170] The first acquisition module is used to acquire the image detection model to be trained and the sample image set; the image detection model to be trained is obtained by training a preset neural network model on an image classification task.

[0171] The first sample feature extraction module is used to extract features from each sample image in the sample image set based on the feature extraction network of the image detection model to be trained, so as to obtain the sample image features of each sample image.

[0172] The probability density determination module is used to determine the probability density corresponding to each sample image feature based on the sample image features corresponding to each sample image of the preset category, and to sample the target sample image features based on the probability density corresponding to each sample image feature; the probability density corresponding to the target sample image feature is less than the preset probability density threshold.

[0173] The sample feature mapping module is used to perform feature mapping on each sample image feature based on the fully connected network of the image detection model to be trained, and obtain the sample feature mapping result corresponding to each sample image feature; the sample feature mapping result includes the response value of the corresponding sample image feature to each of the multiple preset categories;

[0174] The sample uncertainty calculation module is used to calculate the uncertainty of the response value of each of the multiple preset categories based on each feature of the sample image, and obtain the prediction uncertainty corresponding to each feature of the sample image;

[0175] The training submodule is used to train the image detection model to be trained with the goal of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty, until the preset training termination condition is met to obtain the image detection model; wherein, the first comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each feature of the sample image, and the second comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each feature of the target sample image.

[0176] In one exemplary embodiment, the probability density determination module includes:

[0177] The first determining module is used to determine the mean value of the sample image features of the preset category for each preset category to obtain the feature mean of the preset category; and to determine the feature covariance of the preset category based on the difference between the sample image features of each preset category and the feature mean of the preset category.

[0178] The second determining module is used to obtain a covariance matrix based on the sum of the feature covariances of each of the multiple preset categories;

[0179] The probability density determination submodule is used to determine the probability density corresponding to each sample image feature based on the covariance matrix and the difference between the sample image feature and the feature mean of the preset category corresponding to the sample image feature.

[0180] In one exemplary implementation, each sample image in the sample image set corresponds to label information, which indicates the preset category corresponding to the corresponding sample image among the multiple preset categories; the training submodule includes:

[0181] An uncertainty loss construction module is used to construct uncertainty loss with the objective of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty;

[0182] The prediction and classification module is used to normalize the sample feature mapping results corresponding to the features of each sample image to obtain the prediction and classification result for each sample image; the prediction and classification result represents the probability that the corresponding sample image belongs to each preset category.

[0183] The classification loss determination module is used to determine the classification loss based on the predicted classification result corresponding to each sample image and the label information of each sample image;

[0184] The comprehensive loss determination module is used to perform a weighted summation of the uncertain loss and the classified loss to obtain the comprehensive loss.

[0185] The parameter adjustment module is used to adjust the model parameters of the image detection model to be trained based on the comprehensive loss until the preset training termination condition is met to obtain the image detection model.

[0186] In one exemplary embodiment, the device further includes:

[0187] The second acquisition module is used to acquire the training image set;

[0188] The image detection model determination module is used to train the preset neural network model on the training image set to obtain the image detection model to be trained; the image classification task is to predict the probability that the training images in the training image set belong to each preset category.

[0189] In one exemplary implementation, the uncertainty calculation is based on an energy function.

[0190] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0191] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement any of the image detection methods provided in the above method embodiments.

[0192] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0193] The method embodiments provided in this application can be executed in a computer terminal, server or similar computing device, that is, the above-mentioned electronic device may include a computer terminal, server or similar computing device. Figure 9 This is a hardware structure block diagram of an electronic device that runs an image detection method according to an embodiment of this application, such as... Figure 9 As shown, the electronic device 90 may include one or more processors 902 (shown as 902a, 902b, ..., 902n in the figure) 902 (processor 902 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 904 for storing data, and a transmission device 906 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 9 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 90 may also include... Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown.

[0194] It should be noted that the aforementioned one or more processors 902 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within the electronic device 90 (or mobile device). As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0195] The memory 904 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of the present invention. The processor 902 executes various functional applications and data processing by running the software programs and modules stored in the memory 904, thereby realizing the above-described image detection method. The memory 904 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 904 may further include memory remotely located relative to the processor 902, and these remote memories can be connected to the electronic device 90 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0196] The transmission device 906 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 90. In one example, the transmission device 906 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one embodiment, the transmission device 906 may be a radio frequency (RF) module for wireless communication with the Internet.

[0197] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of an electronic device (or mobile device).

[0198] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing an image detection method. The at least one instruction or the at least one program is loaded and executed by the processor to implement any of the image detection methods provided in the above-described method embodiments.

[0199] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the image detection methods provided in the above-described method embodiments.

[0200] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0201] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0202] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0203] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0204] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image detection method, characterized in that, The method includes: The image to be detected is acquired, and the image to be detected is input into the feature extraction network of the image detection model to extract features, thereby obtaining the image features corresponding to the image to be detected. The image features are input into the fully connected network of the image detection model for feature mapping to obtain the feature mapping result; the feature mapping result includes the response value of the image features to each of the multiple preset categories; Using an energy function, energy is calculated for the response value of each of the multiple preset categories based on the image features to obtain the uncertainty of the image to be detected; the uncertainty characterizes the degree of uncertainty of the image to be detected belonging to the multiple preset categories; When the uncertainty exceeds a preset uncertainty threshold, a first detection result is output; the first detection result indicates that the image to be detected is an abnormal input image. The method further includes training the image detection model, wherein training the image detection model includes: Obtain the image detection model to be trained and the sample image set; the image detection model to be trained is obtained by training a preset neural network model on an image classification task. Based on the feature extraction network of the image detection model to be trained, feature extraction is performed on each sample image in the sample image set to obtain the sample image features of each sample image; Based on the sample image features corresponding to each of the preset categories, the probability density corresponding to each sample image feature is determined, and the target sample image features are sampled based on the probability density corresponding to each sample image feature; the probability density corresponding to the target sample image feature is less than a preset probability density threshold. The fully connected network based on the image detection model to be trained performs feature mapping on each sample image feature to obtain the sample feature mapping result corresponding to each sample image feature; the sample feature mapping result includes the response value of the corresponding sample image feature to each of the multiple preset categories; Using the energy function, the energy of the response value of each of the multiple preset categories is calculated based on each of the sample image features to obtain the prediction uncertainty corresponding to each of the sample image features; The image detection model to be trained is trained with the goal of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty until a preset training termination condition is met to obtain the image detection model; wherein, the first comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each of the sample image features, and the second comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each of the target sample image features.

2. The method according to claim 1, characterized in that, The method further includes: When the uncertainty does not exceed the preset uncertainty threshold, the feature mapping result is normalized to obtain a normalized result; the normalized result represents the probability that the image to be detected belongs to each of the multiple preset categories; Based on the normalization result, the preset category corresponding to the highest probability is selected as the target category of the image to be detected. Based on the target category of the image to be detected, a second detection result is output; the second detection result indicates the probability that the image to be detected belongs to the target category.

3. The method according to claim 1, characterized in that, The step of determining the probability density corresponding to each sample image feature based on the sample image features corresponding to each preset category includes: For each preset category, the mean value of the sample image features of the preset category is determined to obtain the feature mean of the preset category; based on the difference between each sample image feature of the preset category and the feature mean of the preset category, the feature covariance of the preset category is determined. Based on the sum of the feature covariances of each of the multiple preset categories, a covariance matrix is ​​obtained; For each of the sample image features, the probability density corresponding to the sample image feature is determined based on the covariance matrix and the difference between the sample image feature and the feature mean of the preset category corresponding to the sample image feature.

4. The method according to claim 1, characterized in that, Each sample image in the sample image set corresponds to a label information, and the label information indicates the preset category that the corresponding sample image belongs to in the plurality of preset categories; The step of training the image detection model to be trained with the objective of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty until a preset training termination condition is met to obtain the image detection model includes: An uncertainty loss is constructed with the objective of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty. The sample feature mapping result corresponding to each sample image feature is normalized to obtain the prediction classification result corresponding to each sample image; the prediction classification result represents the probability that the corresponding sample image belongs to each of the preset categories; Based on the predicted classification result and the label information of each sample image, the classification loss is determined; The combined loss is obtained by weighted summation of the uncertainty loss and the classification loss. The model parameters of the image detection model to be trained are adjusted in reverse based on the comprehensive loss until the preset training termination condition is met to obtain the image detection model.

5. The method according to claim 1, characterized in that, Before obtaining the image detection model to be trained, the method further includes: Obtain the training image set; The preset neural network model is trained on the training image set to perform an image classification task, thereby obtaining the image detection model to be trained; the image classification task is to predict the probability that the training images in the training image set belong to each of the preset categories.

6. An image detection device, characterized in that, The device includes: The image feature extraction module is used to acquire the image to be detected, input the image to be detected into the feature extraction network of the image detection model for feature extraction, and obtain the image features corresponding to the image to be detected. The feature mapping module is used to input the image features into the fully connected network of the image detection model for feature mapping to obtain the feature mapping result; the feature mapping result includes the response value of the image features to each of the multiple preset categories; An uncertainty calculation module is used to calculate the energy of the response value of each of the multiple preset categories based on the image features using an energy function, thereby obtaining the uncertainty of the image to be detected; the uncertainty characterizes the degree of uncertainty of the image to be detected belonging to the multiple preset categories; The first detection result output module is used to output a first detection result when the uncertainty exceeds a preset uncertainty threshold; the first detection result indicates that the image to be detected is an abnormal input image; A training module for training the image detection model, the training module comprising: The first acquisition module is used to acquire the image detection model to be trained and the sample image set; the image detection model to be trained is obtained by training a preset neural network model on an image classification task. The first sample feature extraction module is used to extract features from each sample image in the sample image set based on the feature extraction network of the image detection model to be trained, so as to obtain the sample image features of each sample image. The probability density determination module is used to determine the probability density corresponding to each sample image feature based on the sample image features corresponding to each sample image of the preset category, and to sample target sample image features based on the probability density corresponding to each sample image feature; the probability density corresponding to the target sample image feature is less than a preset probability density threshold. The sample feature mapping module is used to perform feature mapping on each sample image feature based on the fully connected network of the image detection model to be trained, so as to obtain the sample feature mapping result corresponding to each sample image feature; the sample feature mapping result includes the response value of the corresponding sample image feature to each of the multiple preset categories; The sample uncertainty calculation module is used to calculate the uncertainty of the response value of each of the plurality of preset categories based on each of the sample image features, so as to obtain the prediction uncertainty corresponding to each of the sample image features; The training submodule is used to train the image detection model to be trained with the goal of minimizing the first comprehensive prediction uncertainty and maximizing the second comprehensive prediction uncertainty, until the preset training termination condition is met to obtain the image detection model; wherein, the first comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each of the sample image features, and the second comprehensive prediction uncertainty is the sum of the prediction uncertainties corresponding to each of the target sample image features.

7. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the image detection method as described in any one of claims 1 to 5.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the image detection method according to any one of claims 1 to 5.

Images