Image anomaly detection method, image anomaly detection model training method and device

Abstract

The application provides an image anomaly detection method and an image anomaly detection model training method and device, relates to the technical field of machine learning, and is used for solving the problem of low accuracy of an image anomaly detection model in detecting anomalies. The method comprises the following steps: acquiring a first image and a first information library. The first image is input into a trained feature recognition model to obtain a plurality of second feature information, and the feature recognition model is used for recognizing feature information in the image. The plurality of second feature information is input into a trained feature reconstruction model to obtain a second information library, and the feature reconstruction model is used for splicing feature information according to feature positions. The second information library comprises a plurality of third feature information. The first information library and the second information library are input into a trained detection model to generate detection information, the detection model is used for detecting whether the two information libraries have feature information with the same feature positions and different feature parameters, and the detection information is used for indicating the state of a to-be-detected object.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to an image anomaly detection method, an image anomaly detection model training method and apparatus. Background Technology

[0002] In recent years, with the development of image recognition technology, it has been widely applied in fields such as medicine, entertainment, and industry. For example, servers use image anomaly detection models to detect anomalies in device images.

[0003] Currently, in the process of a server detecting device anomalies using an image anomaly detection model, the server first needs to acquire abnormal images of the device and then train the image anomaly detection model based on these images. Afterward, the server uses the trained image anomaly detection model to detect anomalies in the device images to determine if the device is abnormal. However, in this technical solution, when the number of abnormal device images is small, the accuracy of the image anomaly detection model in detecting anomalies may be low. Summary of the Invention

[0004] This application provides an image anomaly detection method, an image anomaly detection model training method, and an apparatus to address the problem of low accuracy in anomaly detection by image anomaly detection models.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, this application provides an image anomaly detection method, comprising: an image anomaly detection device (hereinafter referred to as a "detection device") acquiring a first image and a first information database, wherein the first image is an image of an object to be detected, and the first information database includes: multiple first feature information, wherein the first feature information is feature information of the object to be detected in a normal state, and the feature information includes: feature position and feature parameters. The detection device inputs the first image into a trained feature recognition model to obtain multiple second feature information, and the feature recognition model is used to identify the feature information in the image. The detection device inputs the multiple second feature information into a trained feature reconstruction model to obtain a second information database, and the feature reconstruction model is used to concatenate feature information according to feature positions. The second information database includes: multiple third feature information, wherein the feature positions of the second feature information in a third feature information are all the same. The detection device inputs the first information database and the second information database into the trained detection model to generate detection information, and the detection model is used to detect whether there are feature information in the two information databases that have the same feature position but different feature parameters, and the detection information is used to indicate the state of the object to be detected.

[0007] Understandably, the detection device identifies the image of the object to be detected using an image anomaly detection model, obtaining feature information from multiple different locations. Then, based on the feature positions in each feature information, the detection device stitches together these multiple feature information entries using the image anomaly detection model, resulting in multiple stitched feature information entries. Next, the detection device compares each stitched feature information entry with the feature information of the object under normal conditions using the image anomaly detection model to detect whether the object is abnormal. In other words, the image anomaly detection model can determine whether the object is abnormal based on the feature information of the object under normal conditions from the image of the object. This improves the applicability and accuracy of the image anomaly detection model in detecting anomalies.

[0008] Optionally, the feature recognition model includes: multiple first network layers, each first network layer being used to recognize feature information in one dimension of an image. The method described above, where "the detection device inputs a first image into the trained feature recognition model to obtain multiple second feature information," includes: for the first image, the detection device determines each second feature information through a first operation, the first operation including: inputting the first image into a second network layer to obtain at least one fourth feature information, where the second network layer is any one of the trained multiple first network layers, and the fourth feature information is the feature information in the dimension recognized by the second network layer among the multiple second feature information.

[0009] Optionally, if the trained detection model detects that the first and second information databases contain feature information with the same feature location but different feature parameters, the specific detection information is: the object to be detected is in an abnormal state. If the trained detection model detects that the first and second information databases do not contain feature information with the same feature location but different feature parameters, the specific detection information is: the object to be detected is in a normal state.

[0010] Optionally, the detection model is also used to label feature information in the two information databases that have the same feature location but different feature parameters. The image anomaly detection method further includes: if the detection information indicates that the object to be detected is in an abnormal state, the detection device inputs the second information database into the trained image reconstruction model to obtain a second image. The image reconstruction model is used to generate an image based on the feature information, and the second image includes the third feature information that has been labeled in the second information database.

[0011] Secondly, this application provides a method for training an image anomaly detection model. The method includes: an image anomaly detection model training device (hereinafter referred to as a "training device") acquiring a training set, the training set including: a preset image and multiple first preset feature information, wherein the preset image is an image of the object to be detected, and the first preset feature information is feature information in the preset image, the feature information including: feature location and feature parameters. The training device trains the image anomaly detection model based on the preset image and multiple first preset feature information until the image anomaly detection model converges, obtaining the trained image anomaly detection model. The image anomaly detection model includes: a feature reconstruction model and an image reconstruction model, wherein the feature reconstruction model is used to concatenate feature information according to feature locations, and the image reconstruction model is used to generate an image based on the feature information.

[0012] Understandably, the detection device, based on a preset image and multiple first preset feature information, adjusts the system function of the feature reconstruction model, the parameters of the feature reconstruction network, the system function of the image reconstruction model, and the parameters of the image reconstruction network to make the target loss value less than or equal to a preset loss threshold, thereby training the image anomaly detection model and improving the accuracy of the detection results of the image anomaly detection model.

[0013] Optionally, the feature information also includes: feature dimension. The method described above, "training an image anomaly detection model based on a preset image and multiple first preset feature information," includes: for the image anomaly detection model, the training device trains the image anomaly detection model through a second operation, the second operation including: inputting multiple first preset feature information into a feature reconstruction model to obtain a training information library, the training information library including: multiple first training feature information, each first training feature information including at least one second training feature information, and the feature positions of the second training feature information within a first training feature information are all the same. For each first training feature information, a first feature loss value is determined for each second training feature information within the first training feature information to obtain a second feature loss value. Wherein, the first feature loss value is the loss value between the second training feature information and a fifth feature information among the multiple first preset feature information, the feature positions and feature dimensions of the second training feature information and the fifth feature information are all the same, and the second feature loss value is the sum of the first feature loss values ​​of all second training feature information in the training information library. The training information library is input into the image reconstruction model to obtain a training image. A first image loss value is determined, the first image loss value being the loss value between the training image and the preset image. A target loss value is determined, which is the sum of the second feature loss value and the first image loss value. If the target loss value is greater than a preset loss threshold, the training device re-executes the second operation. If the target loss value is less than or equal to the preset loss threshold, the training device obtains the trained image anomaly detection model.

[0014] Thirdly, this application provides an image anomaly detection device, which includes an acquisition module and a processing module.

[0015] The module includes an acquisition module for acquiring a first image and a first information database. The first image is an image of the object to be detected. The first information database includes multiple first feature information items, which are feature information of the object to be detected in its normal state. These feature information items include feature location and feature parameters. The processing module inputs the first image into a trained feature recognition model to obtain multiple second feature information items. The feature recognition model is used to recognize the feature information in the image. The processing module also inputs the multiple second feature information items into a trained feature reconstruction model to obtain a second information database. The feature reconstruction model is used to concatenate feature information based on feature location. The second information database includes multiple third feature information items, where the feature locations of the second feature information items within a single third feature information item are all identical. The processing module further inputs the first and second information databases into a trained detection model to generate detection information. The detection model is used to detect whether there are feature information items in the two information databases that have the same feature location but different feature parameters. The detection information is used to indicate the state of the object to be detected.

[0016] Optionally, the feature recognition model includes: multiple first network layers, each used to recognize feature information in one dimension of an image. The processing module is specifically used to determine each second feature information for the first image through a first operation. The first operation includes: inputting the first image into a second network layer to obtain at least one fourth feature information, where the second network layer is any one of the trained multiple first network layers, and the fourth feature information is the feature information in the dimension recognized by the second network layer among the multiple second feature information.

[0017] Optionally, if the trained detection model detects that the first and second information databases contain feature information with the same feature location but different feature parameters, the specific detection information is: the object to be detected is in an abnormal state. If the trained detection model detects that the first and second information databases do not contain feature information with the same feature location but different feature parameters, the specific detection information is: the object to be detected is in a normal state.

[0018] Optionally, the detection model is also used to label feature information in the two information databases that have the same feature location but different feature parameters. The processing module is also used to input the second information database into the trained image reconstruction model if the detection information indicates that the object to be detected is in an abnormal state, to obtain a second image. The image reconstruction model is used to generate an image based on the feature information, and the second image includes the third feature information that has been labeled in the second information database.

[0019] Fourthly, this application provides an image anomaly detection model training device, which includes an acquisition module and a processing module.

[0020] The acquisition module is used to acquire a training set, which includes: a preset image and multiple first preset feature information. The preset image is an image of the object to be detected, and the first preset feature information is feature information in the preset image, including feature location and feature parameters. The processing module is used to train an image anomaly detection model based on the preset image and multiple first preset feature information until the image anomaly detection model converges, obtaining the trained image anomaly detection model. The image anomaly detection model includes: a feature reconstruction model and an image reconstruction model. The feature reconstruction model is used to concatenate feature information according to feature locations, and the image reconstruction model is used to generate an image based on the feature information.

[0021] Optionally, the feature information also includes: feature dimension. The processing module is specifically used to train the image anomaly detection model through a second operation. The second operation includes: inputting multiple first preset feature information into the feature reconstruction model to obtain a training information library. The training information library includes: multiple first training feature information, each first training feature information including at least one second training feature information, where the feature positions of the second training feature information within a first training feature information are all the same. For each first training feature information, determining a first feature loss value for each second training feature information within the first training feature information to obtain a second feature loss value. Wherein, the first feature loss value is the loss value between the second training feature information and a fifth feature information among the multiple first preset feature information, where the feature positions and feature dimensions of the second training feature information and the fifth feature information are all the same, and the second feature loss value is the sum of the first feature loss values ​​of all second training feature information in the training information library. Inputting the training information library into the image reconstruction model to obtain a training image. Determining a first image loss value, which is the loss value between the training image and a preset image. Determining a target loss value, which is the sum of the second feature loss value and the first image loss value. The processing module is also used to re-execute the second operation if the target loss value is greater than a preset loss threshold. The processing module is also used to obtain the trained image anomaly detection model if the target loss value is less than or equal to the preset loss threshold.

[0022] Fifthly, this application provides an image anomaly detection device, the device comprising: a processor and a memory coupled together, the memory for storing one or more programs, the one or more programs including computer-executable instructions, wherein when the image anomaly detection device is running, the processor executes the computer-executable instructions stored in the memory to implement any of the image anomaly detection methods described in the first aspect above.

[0023] In a sixth aspect, this application provides an image anomaly detection model training device, the device comprising: a processor and a memory coupled together, the memory for storing one or more programs, the one or more programs including computer-executable instructions, wherein when the image anomaly detection model training device is running, the processor executes the computer-executable instructions stored in the memory to implement any of the optional image anomaly detection model training methods described in the second aspect above.

[0024] In a seventh aspect, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform any of the image anomaly detection methods described in the first aspect or any of the image anomaly detection model training methods described in the second aspect.

[0025] Eighthly, this application provides a computer program product applied to a server. The computer program product includes computer instructions. When the computer instructions are executed on the server, the server implements any of the image anomaly detection methods described in the first aspect and any of the image anomaly detection model training methods described in the second aspect.

[0026] The technical problems that the image anomaly detection device, the image anomaly detection model training device and equipment, the computer storage medium or the computer program product can solve and the technical effects they achieve can be found in the first and second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0027] Figure 1 A schematic diagram of a communication system provided in an embodiment of this application;

[0028] Figure 2 A flowchart illustrating an image anomaly detection model training method provided in an embodiment of this application;

[0029] Figure 3 A flowchart illustrating an image anomaly detection method provided in an embodiment of this application;

[0030] Figure 4 A schematic diagram illustrating an example of image anomaly detection provided in this application embodiment;

[0031] Figure 5 This is a schematic diagram of the structure of an image anomaly detection device provided in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of the structure of an image anomaly detection model training device provided in an embodiment of this application;

[0033] Figure 7 This is a schematic diagram of the structure of an image anomaly detection device provided in an embodiment of this application;

[0034] Figure 8 This is a schematic diagram of the structure of an image anomaly detection model training device provided in an embodiment of this application;

[0035] Figure 9 A conceptual partial view of a computer program product provided for an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] In this article, the character " / " generally indicates that the objects before and after it are in an "or" relationship. For example, A / B can be understood as A or B.

[0038] The terms “first” and “second” in the specification and claims of this application are used to distinguish different objects, rather than to describe a specific order of objects.

[0039] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include other steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or device.

[0040] Furthermore, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the words "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0041] Before providing a detailed introduction to the image anomaly detection method and image anomaly detection model training method provided in the embodiments of this application, the implementation environment and application scenarios of the embodiments of this application will be introduced first.

[0042] First, the application scenarios of the embodiments of this application will be introduced.

[0043] The image anomaly detection method and image anomaly detection model training method of this application are applied in image detection scenarios. In related technologies, during the process of a server performing anomaly detection on a device using an image anomaly detection model, the server first needs to acquire anomaly images of the device and train the image anomaly detection model based on these images. Then, the server uses the trained image anomaly detection model to detect images of the device to determine whether the device is abnormal.

[0044] For example, the server acquires multiple abnormal images of router A, including image A (signal indicator light is red), image B (signal indicator light is off), and image C (power indicator light is off). The server then trains an image anomaly detection model based on images A, B, and C. Next, the server acquires image D of router A to be detected. If the server detects an image identical to image C in image D using the trained image anomaly detection model, the server determines that router A is in an abnormal state.

[0045] In summary, current technical solutions may lead to low accuracy in detecting anomalies when the number of abnormal images on the device is small.

[0046] To address the aforementioned problems, this application provides an image anomaly detection method. The detection device identifies an image of a target object using an image anomaly detection model, obtaining feature information from multiple different locations. Then, based on the feature positions in each feature information, the detection device concatenates the multiple feature information pieces using the image anomaly detection model, obtaining multiple concatenated feature information pieces. Next, the detection device compares each concatenated feature information piece with the feature information of the target object in its normal state using the image anomaly detection model to detect whether the target object is abnormal. In other words, the image anomaly detection model can determine whether the target object is abnormal based on the feature information of the target object in its normal state through the image of the target object. This improves the applicability and accuracy of the image anomaly detection model in detecting anomalies.

[0047] The implementation environment of the embodiments of this application is described below.

[0048] like Figure 1 The diagram shown is a schematic representation of a communication system provided in an embodiment of this application. The communication system may include: a data acquisition device (such as terminal 101), a detection device (such as server 102), and a management device (such as terminal 103). Terminal 101 can communicate with server 102 via wired / wireless communication, and terminal 103 can also communicate with server 102 via wired / wireless communication.

[0049] Terminal 101 can acquire an image of the object to be detected and send the acquired image to server 102. Server 102 can receive the image from terminal 101, perform anomaly detection on the image, and generate a detection result. Then, server 102 can send the detection result to terminal 103. Terminal 103 can receive the detection result from server 102 and present the detection result.

[0050] In some embodiments, the communication system may consist only of a detection device (such as server 102). Server 102 can receive user input commands, acquire images, perform anomaly detection on the acquired images, and generate detection results. Server 102 can then present the detection results.

[0051] In other words, in this embodiment of the application, the communication system may not include acquisition equipment and management equipment.

[0052] It should be noted that, in the embodiments of this application, the server (such as server 102) can be a single physical server, or it can be a server cluster consisting of multiple servers. Alternatively, the server cluster can be a distributed cluster. Or, the server can be a cloud server. The embodiments of this application do not limit the specific implementation method of the server.

[0053] The terminal (such as terminal 101, terminal 103) can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, or other device with sending and receiving functions. This application does not impose any special restrictions on the specific form of the terminal. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device.

[0054] In this embodiment, server 102 may include a feature recognition model 104, a feature reconstruction model 105, and a detection model 106. The feature recognition model 104 is used to recognize feature information in an image, the feature reconstruction model 105 is used to concatenate feature information based on feature locations, and the detection model 106 is used to detect whether two information databases are identical.

[0055] Server 102 can input the received image into feature recognition model 104, and use feature recognition model 104 to identify multiple feature information in the image. Then, server 102 inputs the identified multiple feature information into feature reconstruction model 105, and according to the position of each feature information in the image, the feature reconstruction model 105 concatenates the multiple feature information to generate an information library. Afterwards, server 102 can input the generated information library into detection model 106, and use detection model 106 to check whether the generated information library is the same as a preset information library, generating a detection result.

[0056] It should be noted that the process by which the feature recognition model 104 identifies feature information in an image can be referenced from the conventional technique of identifying feature information in an image through a pre-trained network, which will not be elaborated here.

[0057] In this embodiment, the server 102 may further include an image reconstruction model 107. The image reconstruction model 107 can be used to generate an image based on feature information, and the detection model 106 can be used to label the feature information.

[0058] If the generated information database differs from the preset information database, server 102 can use detection model 106 to mark the different feature information in the generated information database. Then, server 102 can input the marked information database into image reconstruction model 107 and generate an image with the marked feature information using image reconstruction model 107. Server 102 can then present the image with the marked feature information. Alternatively, server 102 can send the image with the marked feature information to terminal 103. Terminal 103 can receive the image with the marked feature information from server 102 and present the image with the marked feature information.

[0059] It should be noted that the process by which the image reconstruction model 107 generates an image based on feature information can be referred to in the introduction of image generation based on information in conventional techniques, and will not be elaborated here.

[0060] After introducing the application scenarios and implementation environment of the embodiments of this application, the image anomaly detection model training method provided by the embodiments of this application will be described in detail below in conjunction with the above implementation environment.

[0061] The methods described in the following embodiments can all be implemented in the above-described application scenarios and implementation environments. The embodiments of this application will be specifically described below using the detection device as a server, in conjunction with the accompanying drawings.

[0062] Figure 2 This is a flowchart illustrating an image anomaly detection model training method provided in an embodiment of this application. Figure 2As shown, the method may include: S201-S203.

[0063] S201, The server retrieves the training set.

[0064] The training set may include: a preset image and multiple first preset feature information. The preset image is an image of the object to be detected, and the first preset feature information is the feature information in the preset image, which may include: feature location and feature parameters.

[0065] For example, the training set includes: an image A (i.e., a preset image) of a keyboard (i.e., the object to be detected), feature information A, feature information B, and feature information C (i.e., multiple first preset feature information). Feature information A, feature information B, and feature information C are all feature information of the keycaps in image A, and feature information A is the exit (ESC) key (i.e., feature position) + black (i.e., feature parameter), feature information B is the control (Ctrl) key (i.e., feature position) + white (i.e., feature parameter), and feature information C is the enter key (i.e., feature position) + black.

[0066] In one possible design, the feature information may also include: feature dimensions.

[0067] It should be noted that the embodiments of this application do not limit the feature dimension. For example, the feature dimension can be a color dimension. Another example is that the feature dimension can be a shape dimension. Yet another example is that the feature dimension can be a texture dimension. Another example is that the feature dimension can be an edge dimension. Yet another example is that the feature dimension can be a contour dimension.

[0068] For example, the multiple first preset feature information includes: feature information A, feature information B, and feature information C. Feature information A is position A (i.e., feature position) + RGB color (i.e., feature dimension) (0, 255, 0) (i.e., feature parameter), feature information B is position B + shape (21, 5, 3), and feature information C is position C + contour (2, 2.5, 30). Among them, (0, 255, 0) indicates green, (21, 5, 3) indicates a circle, and (2, 2.5, 30) indicates a circular curve.

[0069] S202. The server trains an image anomaly detection model based on a preset image and multiple first preset feature information.

[0070] In one possible implementation, the image anomaly detection model may include a feature reconstruction model and an image reconstruction model. For the image anomaly detection model, the server can train the model through a second operation until it converges. This second operation may include five steps (i.e., steps one, two, three, four, and five). Steps one, two, three, four, and five are described below.

[0071] Step 1: The server inputs multiple first-preset feature information into the feature reconstruction model to obtain the training information database.

[0072] The training information base includes: multiple first training feature information, each first training feature information including at least one second training feature information, and the feature positions of the second training feature information in a first training feature information are all the same.

[0073] For example, multiple first preset feature information includes: feature information A, feature information B, and feature information C. Feature information A is location A + dimension A + parameter A, feature information B is location B + dimension A + parameter B, and feature information C is location A + dimension B + parameter C. After the server inputs feature information A, feature information B, and feature information C into the feature reconstruction model, the resulting training information library includes: feature information D (i.e., first training feature information) and feature information E (i.e., first training feature information). Feature information D includes: feature information A (i.e., second training feature information) and feature information C (i.e., second training feature information), where feature information D is location A + dimension A + parameter A + dimension B + parameter C. Feature information E includes: feature information B (i.e., second training feature information), where feature information E is location B + dimension A + parameter B.

[0074] In one possible design, the first feature information can be represented by Formula 1.

[0075]

[0076] Among them, R m f is used to represent the first training feature information at feature position m, N is used to represent the number of feature dimensions in the training information base (i.e., the number of second training feature information in the first training feature information), f 1 System functions used to represent the forward encoding of the feature reconstruction model Feature parameters used to represent the fifth feature information with feature position m and feature dimension n among multiple first preset feature information. This is used to represent the network parameters of a feature reconstruction model with feature location m and feature dimension n.

[0077] Step 2: For each first training feature information, the server can determine the first feature loss value for each second training feature information in the first training feature information to obtain the second feature loss value.

[0078] In one possible implementation, the feature reconstruction model can also be used to split feature information according to feature dimensions. The server can input each first training feature information in the training information base into the feature reconstruction model to obtain second training feature information in each first training feature information. Then, for each first training feature information, the server can determine a first feature loss value for each second training feature information in the first training feature information. The first feature loss value is the loss value between the second training feature information and a fifth feature information among multiple first preset feature information, where the second training feature information and the fifth feature information have the same feature position and feature dimension.

[0079] In one possible design, the second training feature information in the first training feature information can be represented by Equation 2.

[0080]

[0081] in, Used to represent R m The feature parameters f of the second training feature information with feature dimension n. 2 System functions used to represent the inverse encoding of feature reconstruction models.

[0082] For example, multiple first training feature information includes: feature information A, feature information B, and feature information C. Feature information A is position A + dimension A parameter A + dimension B parameter C, feature information B is position B + dimension A parameter A, and feature information C is position C + dimension A parameter B + dimension C parameter A. Then, the server obtains feature information D (i.e., second training feature information) and feature information E (i.e., second training feature information) from feature information A, and feature information F (i.e., second training feature information) and feature information G (i.e., second training feature information) from feature information C using the above formula two. Feature information D is position A + dimension A parameter A, feature information E is position A + dimension B parameter C, feature information F is position C + dimension A parameter B, and feature information G is position C + dimension C parameter A.

[0083] In one possible implementation, the server can determine, based on the feature position and feature dimension of the second training feature information, feature information that has the same feature position and feature dimension as the second training feature information from a plurality of first preset feature information, and then use the feature information that has the same feature position and feature dimension as the second training feature information as the fifth feature information, thereby determining the fifth feature information corresponding to each second training feature information. Afterwards, the server can calculate the loss value between the second training feature information and the corresponding fifth feature information to determine the first feature loss value of the second training feature information.

[0084] In one possible design, the first feature loss value of the second training feature information can be represented by Equation 3.

[0085]

[0086] in, The first feature loss value is used to represent the second training feature information with feature location m and feature dimension n.

[0087] For example, the multiple first preset feature information includes: feature information A, feature information B, and feature information C. Feature information A is position A + dimension A, parameter A; feature information B is position A + dimension B, parameter A; and feature information C is position B + dimension A, parameter B. If the second training feature information D is position A + dimension B, parameter D, then the server determines the fifth feature information corresponding to the second training feature information D as feature information B, and determines the first feature loss value of the second training feature information D as parameter D - parameter A.

[0088] In one possible implementation, after the server determines the first feature loss value for each second training feature information, the server can obtain the second feature loss value of the training information database by using the first feature loss value for each second training feature information in the training information database. The second feature loss value is the sum of the first feature loss values ​​of all second training feature information in the training information database.

[0089] In one possible design, the second feature loss value can be represented by Equation 4.

[0090]

[0091] Among them, l v The second feature loss value is used to represent the training information base, and M is used to represent the number of feature locations in the training information base (i.e., the number of first training feature information).

[0092] Step 3: The server inputs the training information database into the image reconstruction model to obtain the training images.

[0093] In one possible implementation, the image reconstruction model can generate a third image corresponding to each first training feature in the training information base. Then, the image reconstruction model can stitch together multiple third images based on the feature positions of each first training feature to generate a training image corresponding to the training information base. The training image includes all the third images.

[0094] In one possible design, the training images can be represented by Equations 5 and 6.

[0095]

[0096]

[0097] in, Used to represent R m The corresponding third image, g is used to represent the system function of the forward encoding of the image reconstruction model, β m The parameters of the image reconstruction network at feature location m are used to represent the image reconstruction model. I1 is used to represent the training image, and M is also used to represent the number of third images.

[0098] Step four: The server determines the first image loss value.

[0099] In one possible implementation, the server can determine a first image loss value for the training image based on a preset image and a training image. The first image loss value is the loss value between the training image and the preset image.

[0100] In one possible design, the first image loss value can be represented by Equation 7.

[0101]

[0102] Among them, l p The first image loss value used to represent the training image. I2 is used to represent the first image loss value calculated from the complete training images, and I2 is used to represent the preset image.

[0103] In another possible implementation, the server can determine a fourth image from the training images that has the same feature positions as the third image, based on a third image in a preset set of images, with one third image corresponding to one fourth image. Then, the server can determine a second image loss value for each third image based on the multiple third images and multiple fourth images; the second image loss value is the loss between the third image and its corresponding fourth image. Finally, the server determines a first image loss value for the training images based on the second image loss value for each third image; the first image loss value includes all the second image loss values.

[0104] In another possible design, the first image loss value can be represented by Equations 8 and 9.

[0105]

[0106]

[0107] in, Used to represent The second image loss value, Used to represent I2 The corresponding fourth image, This is used to represent the first image loss value calculated by summing the second image loss values ​​of each fourth image in the training images.

[0108] Step 5: The server determines the target loss value.

[0109] In one possible implementation, the server can determine the target loss value based on the second feature loss value and the first image loss value, wherein the target loss value is the sum of the second feature loss value and the first image loss value.

[0110] In one possible design, the target loss value can be expressed by Equation 10.

[0111] l = l v +λ0·l p Formula 10.

[0112] Where l represents the target loss value, and λ0 represents the balancing hyperparameter coefficient between the feature reconstruction model and the image reconstruction model.

[0113] In another possible design, the target loss value can be expressed by Equation 11.

[0114]

[0115] Where λ1 represents the balance hyperparameter coefficient between the feature reconstruction model and the image reconstruction model under the complete image, and λ2 represents the balance hyperparameter coefficient between the feature reconstruction model and the image reconstruction model under the partial image.

[0116] In another possible implementation, the server can use the first image loss value of the training images as the target loss value to determine whether the image anomaly detection model has converged.

[0117] In one possible design, the target loss value can be represented by Equation Twelve.

[0118]

[0119] It should be noted that after the server determines the target loss value, the server can compare the target loss value with the preset loss threshold to determine whether the feature reconstruction model and the image reconstruction model have converged, and thus determine whether the image anomaly detection model has converged.

[0120] In some embodiments, if the server determines that the target loss value is greater than a preset loss threshold, the server determines that the image anomaly detection model has not converged and re-executes the second operation described above on the image anomaly detection model.

[0121] In other embodiments, if the server determines that the target loss value is less than or equal to a preset loss threshold, the server determines that the image anomaly detection model has converged and executes S203.

[0122] S203, The server obtains the trained image anomaly detection model.

[0123] Understandably, the server, based on a preset image and multiple first preset feature information, adjusts the system function of the feature reconstruction model, the parameters of the feature reconstruction network, the system function of the image reconstruction model, and the parameters of the image reconstruction network to make the target loss value less than or equal to the preset loss threshold, thereby training the image anomaly detection model and improving the accuracy of the detection results of the image anomaly detection model.

[0124] In some embodiments, the training set may further include: a preset information library, which is an information library corresponding to a preset image, and the preset information library includes: multiple second preset feature information. In step one of the second operation described above, the server can input the multiple second preset feature information from the preset information library into the feature reconstruction model, and obtain multiple third training feature information through reverse encoding by the feature reconstruction model. The feature positions of the third training feature information in a second preset feature information are all the same. Then, the server performs the remaining four steps for each third training feature information.

[0125] Understandably, the server can also train the image anomaly detection model based on a pre-defined information database. This increases the training steps for the image anomaly detection model and improves the operability of the training.

[0126] It should be noted that, in this embodiment, the image anomaly detection model may further include a feature recognition model. During the server's training of the image anomaly detection model, the server may acquire a pre-trained feature recognition model to obtain a trained feature recognition model; alternatively, the server may train the feature recognition model based on a preset image and multiple first preset feature information to obtain a trained feature recognition model.

[0127] In some embodiments, the server can obtain a pre-trained feature recognition model from the target database to obtain a trained feature recognition model.

[0128] In one possible implementation, the feature recognition model may include multiple first network layers, each used to recognize feature information in one dimension of an image, with one first network layer corresponding to one dimension. The server can obtain pre-trained first network layers corresponding to each first network layer from a target database, based on the first network layers in the feature recognition model, to obtain multiple trained first network layers. Then, the server updates the feature recognition model based on the trained multiple first network layers, obtaining a trained feature recognition model, which includes the trained multiple first network layers.

[0129] It should be noted that the feature dimension of the feature information output by the first network layer is the same as the dimension corresponding to the first network layer.

[0130] It should be noted that, in the embodiments of this application, the target database can be the database of the ImageNet project, the first network layer can be a residual network (ResNet), or the first network layer can be a neural network based on the results of the vision transformer (ViT).

[0131] In other embodiments, the server can train the feature recognition model based on a preset image and multiple first preset feature information to obtain a trained feature recognition model. During the server's training of the image anomaly detection model, the second operation may further include two steps (i.e., steps six and seven). Steps six and seven are described below.

[0132] Step six: The server inputs the preset image into the feature recognition model to obtain multiple sixth feature information.

[0133] It should be noted that after the server obtains multiple sixth feature information, for each sixth feature information, the server executes step seven.

[0134] Step 7: The server determines the seventh feature information corresponding to the sixth feature information from multiple first preset feature information.

[0135] Among them, the sixth feature information and the seventh feature information have the same feature position and feature dimension.

[0136] In one possible implementation, the server can determine, based on the feature position and feature dimension of the sixth feature information, the feature information that is identical to the sixth feature information from multiple first preset feature information, and take the feature information that is identical to the sixth feature information from multiple first preset feature information as the seventh feature information, thereby determining the seventh feature information corresponding to each sixth feature information.

[0137] It should be noted that after the seventh feature information corresponding to the sixth feature information, the server can compare each sixth feature information with the corresponding seventh feature information to determine whether the feature recognition model has converged, and thus determine whether the image anomaly detection model has converged.

[0138] In some embodiments, if there is a sixth feature that is different from the corresponding seventh feature among the multiple sixth feature information, the server determines that the image anomaly detection model has not converged and re-executes the second operation described above on the image anomaly detection model.

[0139] For example, multiple sixth feature information includes: feature information A, feature information B, and feature information C. Feature information A has a feature parameter of 253, feature information B has a feature parameter of 221, and feature information C has a feature parameter of 153. If feature information A corresponds to feature information D, feature information B corresponds to feature information E, and feature information C corresponds to feature information F, and feature information D has a feature parameter of 253, feature information E has a feature parameter of 220, and feature information F has a feature parameter of 153, then the server determines that the feature parameter of feature information B is different from the feature parameter of feature information E, and thus determines that the image anomaly detection model has not converged, and re-executes the second operation.

[0140] In other embodiments, if the server determines that each sixth feature information is the same as the corresponding seventh feature information and the server determines that the target loss value is less than or equal to the preset loss threshold, then the server determines that the image anomaly detection model has converged and executes S203.

[0141] It should be noted that the embodiments of this application do not limit the order of steps six and seven, or steps one, two, three, four, and five. For example, the server may execute steps one, two, three, four, and five first, and then execute steps six and seven. Alternatively, the server may execute steps six and seven first, and then execute steps one, two, three, four, and five. Yet another example is that the server may execute steps one, two, three, four, five, six, and seven simultaneously.

[0142] Understandably, the server can input an image into the feature recognition model and determine whether the model has converged by comparing its recognition result with the features already identified in the image. This allows for the training of the feature recognition model, thereby improving its accuracy.

[0143] In this embodiment, the image anomaly detection model may further include a detection model, which may also be used to detect whether two information databases contain feature information with the same feature location but different feature parameters. The server can train the detection model based on multiple first preset feature information to obtain the trained detection model. In the process of the server training the image anomaly detection model, the second operation may further include two steps (i.e., step eight and step nine). Step eight and step nine are described below.

[0144] Step 8: The server generates a third information database and a fourth information database based on multiple first preset feature information.

[0145] The third information database is the same as the fourth information database, and both the third and fourth information databases include all the first preset feature information.

[0146] Step 9: The server inputs the third and fourth information databases into the detection model.

[0147] It should be noted that after the server inputs the third and fourth information databases into the detection model, the server can determine whether the detection model has converged based on the detection results of the detection model on the third and fourth information databases, and thus determine whether the image anomaly detection model has converged.

[0148] In some embodiments, if the detection model detects that the third information library and the fourth information library have the same feature location but different feature parameters, the server determines that the image anomaly detection model has not converged and re-executes the second operation described above on the image anomaly detection model.

[0149] In other embodiments, if the detection model detects that there are no feature information in the third information database and the fourth information database that have the same feature location but different feature parameters, the server determines that the image anomaly detection model has converged and executes S203.

[0150] Understandably, the server can input two identical information databases into the detection model and determine whether the detection model has converged based on its detection results, thus training the detection model. This can improve the accuracy of the detection results from the trained model.

[0151] It should be noted that after obtaining the trained image anomaly detection model, the server can use the trained image anomaly detection model to perform anomaly detection on the image of the object to be detected, thereby determining whether the object to be detected is abnormal.

[0152] Figure 3 This is a schematic flowchart illustrating an image anomaly detection method provided in an embodiment of this application. Figure 3 As shown, the method may include: S301-S304.

[0153] S301, The server obtains the first image and the first information database.

[0154] The first image is the image of the object to be detected.

[0155] It should be noted that the embodiments of this application do not limit the first image. For example, the first image can be an image of the object to be detected that is the same as the preset image. Or, for another example, the first image can be any image of the object to be detected other than the preset image.

[0156] In this embodiment of the application, the first information database includes: a plurality of first feature information, wherein the first feature information is the feature information of the object to be detected in a normal state.

[0157] In one possible implementation, the server can receive a first image and a first information database from the acquisition device.

[0158] In another possible implementation, the server stores multiple preset information databases, with one database corresponding to the identifier of an object. The server can receive a first image from the acquisition device and, based on the identifier of the object to be detected in the first image, select the database from the multiple preset information databases that contains the same identifier as the object to be detected as the first information database, and then obtain the first information database.

[0159] S302. The server inputs the first image into the trained feature recognition model to obtain multiple second feature information.

[0160] The second feature information is the feature information in the first image.

[0161] In one possible implementation, the server can sequentially input the first image into multiple trained first network layers and obtain multiple second feature information based on the output of each trained first network layer.

[0162] It should be noted that the embodiments of this application do not limit the order in which the first image is sequentially input into multiple trained first network layers.

[0163] In one possible design, for a first image, the server can determine each second feature information through a first operation. The first operation includes: the server inputting the first image into a second network layer to obtain at least one fourth feature information. Here, the second network layer is any one of a plurality of trained first network layers, and the fourth feature information is feature information of a dimension identified by the second network layer among the plurality of second feature information, with each fourth feature information having the same feature dimension.

[0164] S303. The server inputs multiple second feature information into the trained feature reconstruction model to obtain the second information database.

[0165] The second information database includes multiple third-feature information.

[0166] In one possible design, a third feature information includes multiple second feature information, and the feature positions of the second feature information in a third feature information are all the same.

[0167] In other words, the third feature information is composed of multiple second feature information where the feature positions are the same.

[0168] S304. The server inputs the first information database and the second information database into the trained detection model to generate detection information.

[0169] In one possible implementation, the detection model can also be used to generate detection information based on the detection results, which is used to indicate the state of the object to be detected.

[0170] It should be noted that the detection information can indicate that the object being detected is in an abnormal state; or, the detection information can indicate that the object being detected is in a normal state.

[0171] In one possible design, if the trained detection model detects that the first information database and the second information database have the same feature location but different feature parameters, then the detection information indicates that the object to be detected is in an abnormal state.

[0172] For example, the first information database includes feature information A, feature information B, and feature information C, and the second information database includes feature information D, feature information E, and feature information F. If feature information A is position A + parameter 251, feature information B is position A + parameter 17, feature information C is position B + parameter 113, feature information D is position A + parameter 251, feature information E is position A + parameter 16, and feature information F is position B + parameter 113, then the detection information is that the object to be detected is in an abnormal state.

[0173] In another possible design, if the trained detection model detects that there are no features in the first information base and the second information base that have the same feature location but different feature parameters, then the detection information is that the object to be detected is in a normal state.

[0174] The technical solution provided by the above embodiments brings at least the following beneficial effects: The server identifies the image of the object to be detected through an image anomaly detection model, obtaining feature information from multiple different locations. Then, based on the feature location in each feature information, the server concatenates the multiple feature information using the image anomaly detection model, obtaining multiple concatenated feature information. Next, the server compares each concatenated feature information with the feature information of the object to be detected in its normal state using the image anomaly detection model to detect whether the object to be detected is abnormal. In other words, the image anomaly detection model can determine whether the object to be detected is abnormal based on the feature information of the object to be detected in its normal state through the image of the object. This improves the applicability and accuracy of the image anomaly detection model in detecting anomalies.

[0175] In some embodiments, the detection model is further used to label feature information in the two information databases that have the same feature location but different feature parameters. If the detection information generated by the trained detection model indicates that the object to be detected is in an abnormal state, the server inputs the second information database into the trained image reconstruction model to obtain a second image, which includes the labeled third feature information from the second information database.

[0176] Understandably, when the server determines that the object to be detected is in an abnormal state, it can mark the abnormal features in the object's image. This provides valuable information for the maintenance and management of the object, improving management efficiency.

[0177] The image anomaly detection method provided in this application will be described below with specific examples, such as... Figure 4As shown, the server can acquire image 401 of the object to be detected. Then, the server inputs image 401 into pre-trained networks (i.e., the first network layer in the trained feature recognition model) 402, pre-trained network 403, and pre-trained network 404 respectively, obtaining feature libraries 405, 406, and 407. Feature library 405 includes feature information (i.e., second feature information) 408, 409, 410, and 411; feature library 406 includes feature information 412, 413, 414, and 415; and feature library 407 includes feature information 416, 417, 418, and 419. Subsequently, the server inputs feature libraries 405, 406, and 407 into the trained view reconstruction model 420 (i.e., the feature reconstruction model) to obtain a latent space representation library (i.e., the second information library) 421. The latent space representation library 421 includes feature information 422, 423, 424, and 425. Specifically, feature information 422 includes feature information 408, 412, and 416; feature information 423 includes feature information 409, 413, and 417; feature information 424 includes feature information 410, 414, and 418; and feature information 425 includes feature information 411, 415, and 419. The server can then use a detection model to determine whether there are anomalous feature information in the latent space representation library 421 and mark the anomalous feature information in the latent space representation library 421. The server then inputs the labeled latent space representation library 421 into the decoder (i.e., the image reconstruction model) 426 to generate an image. If the detection model determines that the feature information 422 in the latent space representation library 421 is abnormal feature information, then the feature information 422 is labeled, and the image generated by the decoder 426 is image 427, which includes the labeled feature information 422.

[0178] The above description primarily focuses on the solutions provided by the embodiments of this application from the perspective of computer devices. It is understood that, in order to achieve the aforementioned functions, the computer device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the image anomaly detection method steps and image anomaly detection model training method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed through hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] This application also provides an image anomaly detection device. This image anomaly detection device can be a computer device, a CPU within the aforementioned computer device, a processing module for image detection within the aforementioned computer device, or a client application for image detection within the aforementioned computer device.

[0180] This application embodiment can divide the image anomaly detection device into functional modules or functional units according to the above method examples. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0181] like Figure 5 The diagram shown is a structural schematic of an image anomaly detection device provided in an embodiment of this application. The image anomaly detection device is used to perform... Figure 3 The image anomaly detection method shown. The image anomaly detection device 500 may include: an acquisition module 501 and a processing module 502.

[0182] The acquisition module 501 is used to acquire a first image and a first information database. The first image is an image of the object to be detected. The first information database includes multiple first feature information, which are feature information of the object to be detected in a normal state. The feature information includes feature location and feature parameters. The processing module 502 is used to input the first image into a trained feature recognition model to obtain multiple second feature information. The feature recognition model is used to recognize the feature information in the image. The processing module 502 is also used to input the multiple second feature information into a trained feature reconstruction model to obtain a second information database. The feature reconstruction model is used to concatenate feature information based on feature location. The second information database includes multiple third feature information, where the feature locations of the second feature information are all the same in a third feature information. The processing module 502 is also used to input the first and second information databases into a trained detection model to generate detection information. The detection model is used to detect whether there are feature information in the two information databases that have the same feature location but different feature parameters. The detection information is used to indicate the state of the object to be detected.

[0183] Optionally, the feature recognition model includes: multiple first network layers, each first network layer being used to recognize feature information in one dimension of an image. The processing module 502 is specifically used to determine each second feature information for the first image through a first operation. The first operation includes: inputting the first image into a second network layer to obtain at least one fourth feature information, wherein the second network layer is any one of the trained multiple first network layers, and the fourth feature information is the feature information in the dimension recognized by the second network layer among the multiple second feature information.

[0184] Optionally, if the trained detection model detects that the first and second information databases contain feature information with the same feature location but different feature parameters, the specific detection information is: the object to be detected is in an abnormal state. If the trained detection model detects that the first and second information databases do not contain feature information with the same feature location but different feature parameters, the specific detection information is: the object to be detected is in a normal state.

[0185] Optionally, the detection model is also used to label feature information in the two information databases that have the same feature location but different feature parameters. The processing module 502 is also used to input the second information database into the trained image reconstruction model if the detection information indicates that the object to be detected is in an abnormal state, to obtain a second image. The image reconstruction model is used to generate an image based on the feature information, and the second image includes the third feature information that has been labeled in the second information database.

[0186] This application also provides an image anomaly detection model training device. This image anomaly detection model training device can be a computer device, a CPU within the aforementioned computer device, a processing module within the aforementioned computer device used for training, or a client within the aforementioned computer device used for training.

[0187] This application embodiment can divide the image anomaly detection model training device into functional modules or functional units according to the above method examples. For example, each function can be divided into its own functional modules or functional units, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module or functional unit. The module or unit division in this application embodiment is illustrative and represents only one logical functional division; other division methods may be used in actual implementation.

[0188] like Figure 6 The diagram shown is a structural schematic of an image anomaly detection model training device provided in an embodiment of this application. The image anomaly detection model training device is used to execute... Figure 2The image anomaly detection model training method is shown. The image anomaly detection model training device 600 may include: an acquisition module 601 and a processing module 602.

[0189] The acquisition module 601 is used to acquire a training set, which includes: a preset image and multiple first preset feature information. The preset image is an image of the object to be detected, and the first preset feature information is feature information in the preset image, including: feature location and feature parameters. The processing module 602 is used to train an image anomaly detection model based on the preset image and multiple first preset feature information until the image anomaly detection model converges, obtaining a trained image anomaly detection model. The image anomaly detection model includes: a feature reconstruction model and an image reconstruction model. The feature reconstruction model is used to concatenate feature information according to feature location, and the image reconstruction model is used to generate an image according to the feature information.

[0190] Optionally, the feature information also includes: feature dimension. Processing module 602 is specifically used to train the image anomaly detection model through a second operation. The second operation includes: inputting multiple first preset feature information into the feature reconstruction model to obtain a training information library. The training information library includes: multiple first training feature information, each first training feature information including at least one second training feature information, where the feature positions of the second training feature information within a first training feature information are all the same. For each first training feature information, determining a first feature loss value for each second training feature information within the first training feature information to obtain a second feature loss value. Wherein, the first feature loss value is the loss value between the second training feature information and a fifth feature information among the multiple first preset feature information, where the feature positions and feature dimensions of the second training feature information and the fifth feature information are all the same, and the second feature loss value is the sum of the first feature loss values ​​of all second training feature information in the training information library. Inputting the training information library into the image reconstruction model to obtain a training image. Determining a first image loss value, which is the loss value between the training image and a preset image. Determining a target loss value, which is the sum of the second feature loss value and the first image loss value. The processing module 602 is further configured to re-execute the second operation if the target loss value is greater than a preset loss threshold. The processing module 602 is also configured to obtain the trained image anomaly detection model if the target loss value is less than or equal to the preset loss threshold.

[0191] Figure 7 This is a schematic diagram of the hardware structure of an image anomaly detection device according to an exemplary embodiment. The image anomaly detection device may include a processor 702, which executes application code to implement the image anomaly detection method of this application.

[0192] The processor 702 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0193] like Figure 7 As shown, the image anomaly detection device may further include a memory 703. The memory 703 stores the application code that executes the scheme of this application, and its execution is controlled by the processor 702.

[0194] Memory 703 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 703 may exist independently and be connected to processor 702 via bus 704. Memory 703 may also be integrated with processor 702.

[0195] like Figure 7 As shown, the image anomaly detection device may further include a communication interface 701, wherein the communication interface 701, processor 702, and memory 703 may be coupled to each other, for example, through a bus 704. The communication interface 701 is used for information exchange with other devices, for example, supporting information exchange between the image anomaly detection device and other devices.

[0196] It should be pointed out that, Figure 7 The device structure shown does not constitute a limitation on the image anomaly detection device, except... Figure 7 In addition to the components shown, the image anomaly detection device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0197] In actual implementation, the functions implemented by processing module 502 can be derived by... Figure 7 The processor 702 shown calls the program code in memory 703 to implement this.

[0198] Figure 8 This is a schematic diagram of the hardware structure of an image anomaly detection model training device according to an exemplary embodiment. The image anomaly detection model training device may include a processor 802, which executes application code to implement the image anomaly detection model training method of this application.

[0199] The processor 802 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0200] like Figure 8 As shown, the image anomaly detection model training device may further include a memory 803. The memory 803 stores the application code that executes the scheme of this application, and its execution is controlled by the processor 802.

[0201] The memory 803 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 803 may exist independently and be connected to the processor 802 via bus 804. The memory 803 may also be integrated with the processor 802.

[0202] like Figure 8As shown, the image anomaly detection model training device may further include a communication interface 801, wherein the communication interface 801, processor 802, and memory 803 may be coupled to each other, for example, through a bus 804. The communication interface 801 is used for information interaction with other devices, for example, supporting information interaction between the image anomaly detection model training device and other devices.

[0203] It should be pointed out that, Figure 8 The device structure shown does not constitute a limitation on the training device for the image anomaly detection model, except... Figure 8 In addition to the components shown, the image anomaly detection model training device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0204] In actual implementation, the functions implemented by processing module 602 can be derived by... Figure 8 The processor 802 shown calls the program code in memory 803 to implement this.

[0205] This application also provides a computer-readable storage medium storing instructions that, when executed by a processor of a computer device, enable the computer to perform the image anomaly detection method and image anomaly detection model training method provided in the embodiments described above. For example, the computer-readable storage medium may be a memory 703 and a memory 803 including instructions, which can be executed by processors 702 and 802 of the computer device to complete the methods described above. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0206] Figure 9 A conceptual partial view of a computer program product provided in an embodiment of this application is shown as an example. The computer program product includes a computer program for executing computer processes on a computing device.

[0207] In one embodiment, the computer program product is provided using a signal bearer medium 900. The signal bearer medium 900 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 2 or Figure 3 The described function or part of the function. Therefore, for example, refer to... Figure 2 In the embodiment shown, one or more features of S201 to S203 can be fulfilled by one or more instructions associated with the signal carrying medium 900. Furthermore, Figure 9 The program instructions in the document also describe example instructions.

[0208] In some examples, the signal carrying medium 900 may include a computer-readable medium 901, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0209] In some implementations, the signal carrying medium 900 may include a computer recordable medium 902, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0210] In some implementations, the signal carrying medium 900 may include a communication medium 903, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0211] The signal-bearing medium 900 can be transmitted by a wireless communication medium 903. One or more program instructions may be, for example, computer-executable instructions or logical implementation instructions.

[0212] In some examples, such as targeting Figure 5 The described image anomaly detection device can be configured to provide various operations, functions, or actions in response to one or more program instructions via a computer-readable medium 901, a computer-recordable medium 902, and / or a communication medium 903.

[0213] In other examples, such as targeting Figure 6 The described image anomaly detection model training apparatus can be configured to provide various operations, functions, or actions in response to one or more program instructions via a computer-readable medium 901, a computer-recordable medium 902, and / or a communication medium 903.

[0214] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0215] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0216] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be in one place or distributed in multiple different locations. Some or all of the constituent units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0217] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0218] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, essentially, or the part that contributes to the prior art, or a complete or partial classification of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0219] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image anomaly detection method, characterized in that, The method includes: Acquire a first image and a first information database. The first image is an image of the object to be detected. The first information database includes: multiple first feature information. The first feature information is the feature information of the object to be detected in a normal state. The feature information includes: feature location and feature parameters. The first image is input into the trained feature recognition model to obtain multiple second feature information, and the feature recognition model is used to recognize the feature information in the image. The multiple second feature information is input into the trained feature reconstruction model to obtain a second information base. The feature reconstruction model is used to concatenate the feature information according to the feature position. The second information base includes: multiple third feature information, in which the feature positions of the second feature information are all the same. The first information database and the second information database are input into the trained detection model to generate detection information. The detection model is used to detect whether there are feature information in the two information databases that have the same feature position and different feature parameters. The detection information is used to indicate the state of the object to be detected. The feature recognition model includes: multiple first network layers, wherein the first network layers are used to recognize the feature information in one dimension of the image; The first image is input into the trained feature recognition model to obtain multiple second feature information, including: For the first image, each piece of the second feature information is determined through a first operation, the first operation including: The first image is input into the second network layer to obtain at least one fourth feature information. The second network layer is any one of the trained plurality of first network layers, and the fourth feature information is the feature information of the dimension identified by the second network layer among the plurality of second feature information.

2. The method according to claim 1, characterized in that, If the trained detection model detects feature information in the first information library and the second information library that has the same feature position but different feature parameters, then the detection information specifically means that the object to be detected is in an abnormal state. If the trained detection model detects that the first information base and the second information base do not contain feature information with the same feature position and different feature parameters, then the detection information specifically means that the object to be detected is in the normal state.

3. The method according to claim 2, characterized in that, The detection model is further used to mark feature information in the two information databases that have the same feature position but different feature parameters; after generating the detection information, the method further includes: If the detection information indicates that the object to be detected is in the abnormal state, then the second information library is input into the trained image reconstruction model to obtain a second image. The image reconstruction model is used to generate an image based on feature information, and the second image includes the third feature information that has been labeled in the second information library.

4. A method for training an image anomaly detection model, characterized in that, The method includes: A training set is obtained, the training set including: a preset image and multiple first preset feature information, the preset image being an image of the object to be detected, the first preset feature information being feature information in the preset image, the feature information including: feature location, feature parameters, and feature dimensions; Based on the preset image and the plurality of first preset feature information, an image anomaly detection model is trained until the image anomaly detection model converges to obtain the trained image anomaly detection model. The image anomaly detection model includes: a feature reconstruction model and an image reconstruction model. The feature reconstruction model is used to splice the feature information according to the feature position, and the image reconstruction model is used to generate an image according to the feature information. The step of training an image anomaly detection model based on the preset image and the plurality of first preset feature information includes: For the image anomaly detection model, the image anomaly detection model is trained through a second operation, the second operation including: The plurality of first preset feature information is input into the feature reconstruction model to obtain a training information library. The training information library includes: a plurality of first training feature information, each of which includes at least one second training feature information. The feature positions of the second training feature information in a first training feature information are all the same. For each of the first training feature information, a first feature loss value is determined for each of the second training feature information in the first training feature information to obtain a second feature loss value; wherein, the first feature loss value is the loss value between the second training feature information and the fifth feature information among the plurality of first preset feature information, the feature position and the feature dimension of the second training feature information and the fifth feature information are the same, and the second feature loss value is the sum of the first feature loss values ​​of all the second training feature information in the training information database; The training information database is input into the image reconstruction model to obtain training images; Determine a first image loss value, where the first image loss value is the loss value between the training image and the preset image; Determine a target loss value, wherein the target loss value is the sum of the second feature loss value and the first image loss value; If the target loss value is greater than the preset loss threshold, then the second operation is repeated; If the target loss value is less than or equal to the preset loss threshold, then the trained image anomaly detection model is obtained.

5. An image anomaly detection device, characterized in that, The device includes: The acquisition module is used to acquire a first image and a first information database. The first image is an image of the object to be detected. The first information database includes: multiple first feature information. The first feature information is the feature information of the object to be detected in a normal state. The feature information includes: feature location and feature parameters. The processing module is used to input the first image into the trained feature recognition model to obtain multiple second feature information, wherein the feature recognition model is used to recognize the feature information in the image; The processing module is further configured to input the plurality of second feature information into the trained feature reconstruction model to obtain a second information library, wherein the feature reconstruction model is configured to concatenate the feature information according to the feature position; the second information library includes: a plurality of third feature information, wherein the feature positions of the second feature information in one third feature information are all the same; The processing module is further configured to input the first information database and the second information database into the trained detection model to generate detection information. The detection model is configured to detect whether there are feature information in the two information databases that have the same feature position and different feature parameters. The detection information is configured to indicate the state of the object to be detected. The feature recognition model includes: multiple first network layers, wherein the first network layers are used to recognize the feature information in one dimension of the image; The processing module is specifically configured to determine each piece of the second feature information for the first image through a first operation, wherein the first operation includes: The first image is input into the second network layer to obtain at least one fourth feature information. The second network layer is any one of the trained plurality of first network layers, and the fourth feature information is the feature information of the dimension identified by the second network layer among the plurality of second feature information.

6. The apparatus according to claim 5, characterized in that, If the trained detection model detects feature information in the first information library and the second information library that has the same feature position but different feature parameters, then the detection information specifically means that the object to be detected is in an abnormal state. If the trained detection model detects that the first information base and the second information base do not contain feature information with the same feature position and different feature parameters, then the detection information specifically means that the object to be detected is in the normal state.

7. The apparatus according to claim 6, characterized in that, The detection model is also used to mark feature information in the two information databases that have the same feature position but different feature parameters; The processing module is further configured to input the second information library into the trained image reconstruction model if the detection information indicates that the object to be detected is in the abnormal state, thereby obtaining a second image. The image reconstruction model is configured to generate an image based on feature information, and the second image includes the third feature information that has been labeled in the second information library.

8. A training device for an image anomaly detection model, characterized in that, The device includes: An acquisition module is used to acquire a training set, the training set including: a preset image and multiple first preset feature information, the preset image being an image of the object to be detected, the first preset feature information being feature information in the preset image, the feature information including: feature location, feature parameters, and feature dimensions; The processing module is used to train an image anomaly detection model based on the preset image and the plurality of first preset feature information until the image anomaly detection model converges, thereby obtaining the trained image anomaly detection model. The image anomaly detection model includes a feature reconstruction model and an image reconstruction model. The feature reconstruction model is used to splice the feature information according to the feature position, and the image reconstruction model is used to generate an image according to the feature information. Specifically, the processing module is used to train the image anomaly detection model through a second operation, wherein the second operation includes: The plurality of first preset feature information is input into the feature reconstruction model to obtain a training information library. The training information library includes: a plurality of first training feature information, each of which includes at least one second training feature information. The feature positions of the second training feature information in a first training feature information are all the same. For each of the first training feature information, a first feature loss value is determined for each of the second training feature information in the first training feature information to obtain a second feature loss value; wherein, the first feature loss value is the loss value between the second training feature information and the fifth feature information among the plurality of first preset feature information, the feature position and the feature dimension of the second training feature information and the fifth feature information are the same, and the second feature loss value is the sum of the first feature loss values ​​of all the second training feature information in the training information database; The training information database is input into the image reconstruction model to obtain training images; Determine a first image loss value, where the first image loss value is the loss value between the training image and the preset image; Determine a target loss value, wherein the target loss value is the sum of the second feature loss value and the first image loss value; The processing module is further configured to re-execute the second operation if the target loss value is greater than a preset loss threshold; The processing module is further configured to obtain the trained image anomaly detection model if the target loss value is less than or equal to the preset loss threshold.

9. An image anomaly detection device, characterized in that, include: Processor and memory; The processor and the memory are coupled; The memory is used to store one or more programs, the one or more programs including computer execution instructions. When the image anomaly detection device is running, the processor executes the computer execution instructions stored in the memory to cause the image anomaly detection device to perform the image anomaly detection method as described in any one of claims 1-3.

10. An image anomaly detection model training device, characterized in that, include: Processor and memory; The processor and the memory are coupled; The memory is used to store one or more programs, the one or more programs including computer execution instructions. When the image anomaly detection model training device is running, the processor executes the computer execution instructions stored in the memory to cause the image anomaly detection model training device to perform the image anomaly detection model training method as described in claim 4.

11. A computer-readable storage medium storing instructions, characterized in that, When the computer executes the instructions, the computer performs the image anomaly detection method as described in any one of claims 1-3 and the image anomaly detection model training method as described in claim 4.

Images