Unsupervised industrial image edge defect detection method and device

Abstract

The invention discloses an unsupervised industrial image edge defect detection method and device. The method comprises the following steps: acquiring a normal sample image set; the normal sample image set comprises a plurality of normal sample images; performing fusion processing on the mask texture image and the normal sample image to obtain an abnormal sample image set; respectively carrying out edge detection on the normal sample image and the abnormal sample image to obtain a first edge region-of-interest image of the normal sample image and a second edge region-of-interest image of the abnormal sample image; and inputting the first edge region-of-interest image and the second edge region-of-interest image into a to-be-trained anomaly detection model for model training to obtain the anomaly detection model. And the anomaly detection model performs anomaly detection on the edge region-of-interest image of the target image to obtain an anomaly probability score graph, and determines whether the target image is an abnormal image according to the anomaly probability score graph. According to the invention, the efficiency and accuracy of industrial image edge defect detection can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial quality inspection, and in particular to an unsupervised industrial image edge defect detection method and device. BACKGROUND

[0002] Anomaly detection (especially edge defect detection) is an indispensable link in industrial production and an important means to ensure product quality. The edge defect detection link usually detects possible structural defects of products after the production link in the factory. Precise and effective edge defect detection technology is very important in the field of industrial quality inspection. Deep neural networks can automatically extract features of input data (such as images) and have strong feature expression ability. More and more industrial quality inspections use deep neural networks. In related technologies, a supervised deep neural network is used to detect edge defects of products. Specifically, a large number of normal samples (i.e., products without edge defects) and a large number of abnormal samples (i.e., products with edge defects) are obtained, and the abnormal samples are labeled by manual labeling. Then, the supervised deep neural network is trained based on the normal samples and the abnormal samples, so as to detect edge defects of products by using the trained model.

[0003] For the training mode of the supervised deep neural network, this mode is highly dependent on the number of abnormal samples and manual labeling. However, in actual production scenarios, the frequency of products with edge defects is usually low. Therefore, the abnormal samples are very rare compared with the normal samples, which leads to a great time and effort cost for collecting the abnormal samples. In addition, the gap between the normal samples and the abnormal samples is usually small. For example, the abnormality of an industrial production circuit board is usually millimeter level or even micrometer level, which brings great difficulty to the manual labeling of the abnormal samples, resulting in inaccurate labeling of the abnormal samples and affecting the performance of the model. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide an unsupervised industrial image edge defect detection method and device to solve the problems of low efficiency and high false detection rate of the existing industrial image edge defect detection method.

[0005] To solve the above technical problems, the embodiments of the present application are implemented as follows:

[0006] In one aspect, the embodiments of the present application provide an unsupervised industrial image edge defect detection method, comprising:

[0007] obtaining a normal sample image set; the normal sample image set includes a plurality of normal sample images;

[0008] obtaining a mask texture image, and generating an abnormal sample image set according to the mask texture image; the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample images have edge defects;

[0009] respectively performing edge detection on the normal sample images and the abnormal sample images to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images;

[0010] inputting the first edge region of interest image and the second edge region of interest image into an abnormal detection model to be trained to perform model training, and obtaining the abnormal detection model.

[0011] In another aspect, an unsupervised industrial image edge defect detection method is provided, including:

[0012] performing edge detection on a target image to obtain a target edge region of interest image corresponding to the target image;

[0013] inputting the target edge region of interest image into a pre-trained abnormal detection model, performing abnormal detection on the target edge region of interest image by using the abnormal detection model, and obtaining an abnormal probability score map corresponding to the target edge region of interest image; the abnormal detection model is trained according to the unsupervised industrial image edge defect detection method in the above aspect;

[0014] determining abnormal information of the target image according to the abnormal probability score map.

[0015] In still another aspect, an unsupervised industrial image edge defect detection device is provided, including:

[0016] a first obtaining module configured to obtain a normal sample image set; the normal sample image set includes a plurality of normal sample images;

[0017] a second obtaining module configured to obtain a mask texture image, and generate an abnormal sample image set according to the mask texture image; the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample images have edge defects;

[0018] a first detecting module configured to respectively perform edge detection on the normal sample images and the abnormal sample images to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images;

[0019] The model training module is configured to input the first edge region of interest image and the second edge region of interest image into an abnormality detection model to be trained to perform model training, and obtain the abnormality detection model.

[0020] In another aspect, the embodiments of the present application provide an unsupervised industrial image edge defect detection device, characterized in that the device comprises:

[0021] The second detection module is configured to perform edge detection on the target image to obtain a target edge region of interest image corresponding to the target image.

[0022] The abnormality detection module is configured to input the target edge region of interest image into a pre-trained abnormality detection model, perform abnormality detection on the target edge region of interest image by using the abnormality detection model, and obtain an abnormality probability score map corresponding to the target edge region of interest image; the abnormality detection model is trained according to the unsupervised industrial image edge defect detection method of the above aspect.

[0023] The determination module is configured to determine abnormality information of the target image according to the abnormality probability score map.

[0024] In another aspect, the embodiments of the present application provide an electronic device, which comprises a processor and a memory electrically connected to the processor, the memory stores a computer program, and the processor is configured to call and execute the computer program from the memory to implement the above-mentioned unsupervised industrial image edge defect detection method.

[0025] In another aspect, the embodiments of the present application provide a computer readable storage medium for storing a computer program, the computer program can be executed by a processor to implement the above-mentioned unsupervised industrial image edge defect detection method.

[0026] By adopting the technical solutions of the embodiments of the present application, the normal sample image set is acquired, the normal sample image set includes a plurality of normal sample images, and the mask texture image is acquired, the mask texture image and the normal sample image are fused to obtain the abnormal sample image set, the abnormal sample image set includes a plurality of abnormal sample images (i.e. images with edge defects) and label information corresponding to the abnormal sample images; the edge of the normal sample image and the edge of the abnormal sample image are detected respectively to obtain the first edge region of interest image corresponding to the normal sample image and the second edge region of interest image corresponding to the abnormal sample image; and then the first edge region of interest image and the second edge region of interest image are input into the abnormal detection model to be trained to obtain the abnormal detection model. It can be seen that, when the abnormal detection model is trained, the model is not directly trained according to the complete sample image, but is trained according to the edge region of interest image (including the first edge region of interest image and the second edge region of interest image) corresponding to the normal sample image and the abnormal sample image respectively. Since the complete sample image includes various image regions irrelevant to the abnormal detection, such as the background region and other regions that do not need to be detected, if the complete sample image is directly used for abnormal detection, the time cost and the calculation cost of the model training will be significantly increased. Therefore, by locating the edge region of interest image in the sample image, the other region images outside the edge region of interest image do not need to be processed in the model training process, and the model training efficiency is greatly improved. In addition, by locating the edge region of interest image that is more likely to have abnormalities in advance, the accuracy of the abnormal detection in the model training process can be improved, and the model performance is improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of one or more embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in one or more embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creating any creative labor.

[0028] Figure 1 is a schematic flow chart of an unsupervised industrial image edge defect detection method according to an embodiment of the present application;

[0029] Figure 2 is a schematic flow chart of an edge region of interest detection algorithm according to an embodiment of the present application;

[0030] Figure 3 is a schematic principle diagram for acquiring a normal sample image set according to an embodiment of the present application;

[0031] Figure 4 is a training process schematic diagram of an anomaly detection model provided by an embodiment of the present application;

[0032] Figure 5 is a schematic flow chart of an unsupervised industrial image edge defect detection method according to another embodiment of the present application;

[0033] Figure 6 is an application process schematic diagram of an anomaly detection model provided by an embodiment of the present application;

[0034] Figure 7 is a schematic block diagram of an unsupervised industrial image edge defect detection apparatus according to an embodiment of the present application;

[0035] Figure 8 is a schematic block diagram of an unsupervised industrial image edge defect detection apparatus according to an embodiment of the present application;

[0036] Figure 9 is a schematic block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application provide an unsupervised industrial image edge defect detection method and apparatus, to solve the problems of low efficiency and high false detection rate of the existing industrial image edge defect detection method.

[0038] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0039] The unsupervised industrial image edge defect detection method provided by the embodiments of the present application can be executed by an electronic device or by software installed in the electronic device. Specifically, the electronic device can be a terminal device or a server device. The terminal device can include a smart phone, a notebook computer, a smart wearable device, a vehicle-mounted terminal, etc., and the server device can include a standalone physical server, a server cluster composed of multiple servers, or a cloud server capable of cloud computing.

[0040] Figure 1 is a schematic flow chart of an unsupervised industrial image edge defect detection method according to an embodiment of the present application, as shown in Figure 1 the method comprises:

[0041] S102, acquire a normal sample image set; the normal sample image set includes a plurality of normal sample images.

[0042] S104, acquire a mask texture image, fuse the mask texture image and the normal sample image to obtain an abnormal sample image set, the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images.

[0043] The abnormal sample image is an image with edge defects, for example, in the scene of producing a glass panel, the edge of the glass panel is prone to product defects such as scratches, edge collapse, dirt, etc. The glass panel with edge defects is regarded as a glass panel with edge defects, and the corresponding image is regarded as an abnormal image (or abnormal sample image) with edge defects.

[0044] The label information is used to represent that the abnormal sample image has edge defects, and the label information can be represented in the form of a matrix. The number of elements in the matrix is the same as the number of image pixels included in the abnormal sample image. The value of each element is used to represent whether the corresponding image pixel is a pixel in the abnormal area. Optionally, 0 represents that the corresponding image pixel is not a pixel in the abnormal area, and 1 represents that the corresponding image pixel is a pixel in the abnormal area. The abnormal area is the image area where the edge defect is located. The fusion mode of the mask texture image and the normal sample image will be described in detail in the following embodiments.

[0045] S106, respectively, edge detection is performed on the normal sample image and the abnormal sample image to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image.

[0046] In this embodiment, the image can be edge detected by an abnormal image edge detection algorithm, for example, an edge region of interest detection algorithm.

[0047] Optionally, after edge detection is performed on the normal sample image and the abnormal sample image, the edge regions corresponding to the normal sample image and the abnormal sample image are obtained. Further, by cropping the detected edge regions from the corresponding normal sample image and abnormal sample image, the first edge region of interest image corresponding to the normal sample image and the second edge region of interest image corresponding to the abnormal sample image can be obtained. That is, other region images that do not belong to the edge region are cropped from the normal sample image and the abnormal sample image.

[0048] Optionally, the execution process of the edge region of interest detection algorithm can include the steps as shown in Figure 2 .

[0049] S1041, Gaussian smoothing is performed on the normal sample image and the abnormal sample image.

[0050] wherein the value of the Gaussian kernel size is not limited, for example, can be 3.

[0051] S1042, using a Canny edge detection operator to detect the image edges for the Gaussian smoothed normal sample image and the abnormal sample image.

[0052] wherein the operator hyperparameters low threshold and high threshold can be set to 50 and 150 respectively, and the aperture size can be set to 3.

[0053] S1043, using a Hough line detection algorithm to detect the positions of the straight lines in the image edge region, and retaining the horizontal (i.e. 0 degrees relative to the image space coordinate axis X axis) and vertical (i.e. 90 degrees relative to the image space coordinate axis X axis) straight lines.

[0054] S1044, for the detected straight lines, performing outward expansion in the horizontal or vertical direction to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image.

[0055] wherein for the horizontal straight line, outward expansion can be performed in the vertical direction, such as upward or downward. For the vertical straight line, outward expansion can be performed in the horizontal direction, such as leftward or rightward. For example, the horizontal straight line is outward expanded by 60 pixels inward (or upward) and outward (or downward) by 10 pixels. The number of outward expanded pixels is not limited and can be pre-set according to the actual application scenario. The region image obtained after the outward expansion of the number of pixels is the first edge region of interest image and the second edge region of interest image.

[0056] S108, inputting the first edge region of interest image and the second edge region of interest image into the abnormal detection model to be trained to perform model training, and obtaining the abnormal detection model.

[0057] By adopting the technical solutions in the embodiments of the present application, the normal sample image set is acquired, the normal sample image set includes a plurality of normal sample images, and the mask texture image is acquired, the mask texture image and the normal sample image are fused to obtain the abnormal sample image set, the abnormal sample image set includes a plurality of abnormal sample images (i.e., images with edge defects) and label information corresponding to the abnormal sample images; edge detection is respectively performed on the normal sample images and the abnormal sample images to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images; and then the first edge region of interest image and the second edge region of interest image are input into the abnormal detection model to be trained to perform model training, and the abnormal detection model is obtained. As can be seen, when training the abnormal detection model, the model is not directly trained according to the complete sample image, but is trained according to the edge region of interest images (including the first edge region of interest image and the second edge region of interest image) corresponding to the normal sample images and the abnormal sample images. Since the complete sample image includes various image regions irrelevant to the abnormal detection, such as a background region and other regions that do not need to be subjected to abnormal detection, if the complete sample image is directly subjected to abnormal detection, the time cost and the calculation cost of model training will be significantly increased. Therefore, by locating the edge region of interest image in the sample image, the other region images outside the edge region of interest image do not need to be processed in the model training process, and the model training efficiency is greatly improved. In addition, by locating the edge region of interest image in advance, which is more prone to have abnormalities, the accuracy of abnormal detection in the model training process can be improved, and the model performance is improved.

[0058] In one embodiment, when the normal sample image set is acquired, the following steps can be performed:

[0059] First, a candidate image set is acquired, the candidate image set includes N candidate normal images; N is an integer greater than 1.

[0060] Second, image features of the N candidate normal images are extracted. For each normal image, an existing any pre-trained model can be used to extract the image features of the normal image. The pre-trained model can be a pre-trained feature extraction model, i.e., a model having a feature extraction capability. For example, an ImageNet pre-trained model (usually a residual neural network ResNet50) is used to extract features of the N candidate normal images. Optionally, the normal image is subjected to feature extraction according to a preset feature vector dimension to obtain image features of the normal image conforming to the preset feature vector dimension. For example, the image features are represented in the form of a feature vector, and the preset feature vector dimension is 2048 dimensions. For each normal image, a 2048-dimensional feature vector is extracted.

[0061] Thirdly, M normal images are selected from the candidate image set as normal sample images according to the image features of the N candidate normal images, to obtain a normal sample image set; the normal sample image set includes M normal sample images; M is an integer greater than 1 and less than N.

[0062] In this embodiment, when the M normal images are selected from the candidate image set as normal sample images, the following steps A1-A2 can be performed:

[0063] In step A1, any normal image of the N candidate normal images is selected as a normal sample image, and the candidate image set and the normal sample image set are updated.

[0064] The selected normal image is added to the normal sample image set, and the updated candidate image set does not include the normal image that has been added to the normal sample image set.

[0065] In step A2, the following steps are repeatedly performed until the normal sample image set includes M normal sample images: according to the image features, the normal image with the largest image feature distance between each normal sample image in the normal sample image set is determined from the updated candidate image set as a normal sample image, and the normal sample image set is updated.

[0066] Optionally, when the steps A1-A2 are performed, they can be performed from the perspective of image features. Specifically, the image features corresponding to the N candidate images form an image feature set, and the image feature set includes N image features. Then, the M image features are selected from the image feature set by calculating the image feature distance, and the normal images corresponding to the M image features are the normal sample images.

[0067] Suppose that the image feature set composed of the image features corresponding to the N candidate normal images is represented as N={f n |n=1,2,…,N}, where f n represents the image feature corresponding to a single normal image, and the image feature set composed of the M selected image features is represented as M, that is, the set M represents the image feature set corresponding to the normal sample images. Initially, the set N includes N image features, and the set M is empty.

[0068] Step A1 can be performed as follows: a image feature is randomly selected from the set N and added to the set M, and the image feature added to the set M is removed from the set N.

[0069] Step A2 can be performed as follows: the image feature distance between each image feature in the set N and each image feature in the set M is calculated, to obtain a feature distance matrix of the set N and the set M. After the calculation, a feature distance vector C∈R|N| where |N| denotes the number of image features in the current set N. Then, the maximum image feature distance is selected from the feature distance vector C e R |N| and the image feature corresponding to the maximum image feature distance is added into set M and removed from set N. Step A2 is repeated until the number of image features included in set M reaches M.

[0070] where the feature distance vector is obtained from the feature distance matrix as follows: after the image feature distance between each image feature in set N and each image feature in set M is calculated, if set M includes multiple image features, then for each image feature in set N, the image feature distance between the current image feature in set N and the image features in set M will be calculated multiple values, the image feature distance between each image feature in set N and each image feature in set M forms a feature distance matrix, each row of the feature distance matrix respectively represents the image feature distance between a single image feature in set N and each image feature in set M, in this case, the minimum image feature distance can be selected from the multiple values in each row as the image feature distance corresponding to each row, i.e. the final image feature distance between a single image feature in set N and each image feature in set M. That is, each row element of the feature distance matrix is compressed into a value, thereby obtaining the feature distance vector.

[0071] Figure 3 is a schematic diagram for performing step A2 in an embodiment of the present application. As shown in Figure 3 , it is assumed that N = 5 and M = 3. That is, 3 normal images are selected from 5 candidate normal images as normal sample images. The 5 candidate normal images are denoted as I1, I2, I3, I4, I5, respectively. First, the image features of the 5 candidate normal images I1, I2, I3, I4, I5 are extracted, and if f n denotes the image features of each candidate normal image, then the image feature set N = {f1, f2, f3, f4, f5} can be obtained. According to the above steps A1-A2, first, step A1 is performed, and a image feature is randomly selected from set N and added to set M, and the image feature added to set M is removed from set N. It is assumed that image feature f2 is selected, then set M = {f2} and set N = {f1, f3, f4, f5} at this time.

[0072] For the current set M and set N, step A2 is performed, first Figure 3The first round of cycles is shown. The image feature distance between each image feature in set N = {f1, f3, f4, f5} and the image feature in set M = {f2} is calculated, i.e. the image feature distance between f1, f3, f4, f5 and f2 respectively, as shown in Figure 3 The values shown are 0.2, 1.2, 0.8, 2.0 respectively. Since set N only includes one image feature at this time, the image feature distance between each image feature in set N and the image feature in set M = {f2} has a unique value. Obviously, the image feature distance between f5 and f2 is the largest, so image feature f5 is added to set M and image feature f5 is removed from set N, resulting in new sets: M = {f2, f5}, N = {f1, f3, f4}.

[0073] Since the number of image features in the current set M has not reached 3, the above actions still need to be repeated, i.e. the second round of cycles is performed as shown in Figure 3 The image feature distance between each image feature in set N = {f1, f3, f4} and the image features in set M = {f2, f5} is calculated, and for each current image feature in set N = {f1, f3, f4}, the minimum value of all image feature distances is selected as the final image feature distance corresponding to the current image feature. For example, for the current image feature f1, the image feature distance between f1 and f2 is calculated to be 0.2, and the image feature distance between f1 and f5 is calculated to be 1.3, so 0.2 is determined as the final image feature distance between the current image feature f1 and the image features in set M = {f2, f5}. Similarly, the final image feature distance between image feature f3 and the image features in set M = {f2, f5} can be calculated to be 0.1, and the final image feature distance between image feature f4 and the image features in set M = {f2, f5} can be calculated to be 0.5. Obviously, in the feature distance vector C = {0.2, 0.1, 0.5}, the image feature distance between f4 and f5 is the largest (i.e. 0.5), so image feature f4 can be added to set M and image feature f4 is removed from set N, resulting in new sets: M = {f2, f5, f4}, N = {f1, f3}.

[0074] At this time, since the number of image features in set M is 3, the cycle can be stopped, and in the resulting set M = {f2, f5, f4}, the normal images corresponding to f2, f5, f4 respectively are the normal sample images.

[0075] In this embodiment, the normal image with the largest distance between image features is selected from the set of candidate images as the normal sample image. That is, each normal sample image screened out has a large distance between image features and other normal sample images. This makes each normal sample image representative, and the normal sample image set can contain richer sample information as much as possible, so that richer and more generalized image features can be learned during the model training process, which greatly reduces the false detection rate of unknown areas and improves the performance of the final trained model.

[0076] In one embodiment, after obtaining a set of normal sample images, a set of abnormal sample images can be obtained based on the normal sample image set. Optionally, a mask texture image corresponding to the abnormal sample image is obtained, and the mask texture image is then fused with the normal sample image to obtain the abnormal sample image corresponding to the normal sample image, as well as label information corresponding to the abnormal sample image.

[0077] Optionally, a texture dataset is pre-acquired, comprising multiple texture images. A random two-dimensional mask is generated using a Perlin noise generation algorithm, thereby generating a corresponding mask texture image based on the texture dataset and the random two-dimensional mask. The mask texture image has a random shape and position. The texture dataset provides texture information for synthesizing the abnormal sample image, while the random two-dimensional mask provides shape and position information.

[0078] The label information is used to indicate that the abnormal sample image has an edge defect. The label information can be represented in a matrix format. The number of elements in the matrix is ​​equal to the number of image pixels in the abnormal sample image. The value of each element indicates whether the corresponding image pixel is in the abnormal region. Optionally, 0 indicates that the corresponding image pixel is not in the abnormal region, and 1 indicates that the corresponding image pixel is in the abnormal region. The abnormal region is the image region where the edge defect is located.

[0079] Alternatively, the fusion of the mask texture image and the normal sample image can be expressed as the following formula:

[0080] I ng =α*M*I texture +(1-α)α*M*I ok +(1-M)*I ok

[0081] Among them, I texture Represents the texture image provided by the texture dataset, I ok represents a normal sample image, M represents a two-dimensional mask generated by the Perlin noise generation algorithm, α represents the intensity of the texture image in the color space, I ng Represents the abnormal sample image obtained after fusion.

[0082] In this embodiment, the abnormal sample image is automatically generated from the normal sample image and the mask texture image, so that the acquisition of the abnormal sample image does not depend on manual operation, and the efficiency and convenience of sample data acquisition are improved. Especially in the scene where the edge defect product appears less frequently, the convenience and efficiency of acquiring the abnormal sample image in this application can be better reflected.

[0083] In one embodiment, the abnormality detection model to be trained includes an encoding network, a feature reconstruction network, a decoding network, and a loss calculation layer. The process of inputting the first edge region of interest image and the second edge region of interest image into the abnormality detection model to be trained for model training can be specifically implemented as the following steps:

[0084] The first edge region of interest image and the second edge region of interest image are feature extracted by the encoding network to obtain a multi-scale normal sample image feature corresponding to the first edge region of interest image and a multi-scale abnormal sample image feature corresponding to the second edge region of interest image. Wherein, multi-scale refers to multi-resolution, the encoding network includes multiple network levels, and different network levels correspond to different resolutions, and each network level can perform feature extraction on the image based on the respective corresponding resolution.

[0085] The abnormal sample image feature is feature reconstructed by the feature reconstruction network to obtain a reconstructed image feature corresponding to the abnormal sample image feature; and the reconstructed image feature and the abnormal sample image feature are fused to obtain a fused image feature. Wherein, the feature reconstruction network includes one or more network levels, and different network levels correspond to different resolutions, if the feature reconstruction network includes only one network level, a single scale (i.e. single resolution) reconstructed image feature can be output, and if the feature reconstruction network includes multiple network levels, a multi-scale (i.e. multi-resolution) reconstructed image feature can be output.

[0086] The decoding network determines an abnormal probability score map corresponding to the second edge region of interest image according to the abnormal sample image feature and the fused image feature; the abnormal probability score map includes an abnormal probability corresponding to each image pixel included in the second edge region of interest image.

[0087] The loss calculation layer calculates a target loss function of the abnormality detection model to be trained according to the normal sample image feature, the reconstructed image feature, the abnormal probability score map, and the label information; and adjusts the parameters of the abnormality detection model to be trained according to the target loss function.

[0088] In one embodiment, when calculating the target loss function of the abnormality detection model to be trained according to the normal sample image feature, the reconstructed image feature, the abnormal probability score map, and the label information, the following steps B1-B3 can be specifically implemented:

[0089] Step B1: determining a first loss function of the anomaly detection model to be trained based on a first difference between the normal sample image features and the reconstructed image features.

[0090] Step B2: determining a second loss function of the anomaly detection model to be trained based on a second difference between the anomaly probability score map and the label information and based on the second difference.

[0091] Step B3: Determine a target loss function based on the first loss function and the second loss function.

[0092] Figure 4 This is a principle diagram of the training process of an anomaly detection model provided by an embodiment of the present application. Figure 4 As shown in the figure, the OK image represents the first edge region of interest image of the normal sample image, and the NG image represents the second edge region of interest image of the abnormal sample image. The NG image carries pre-labeled label information, which is used to indicate that the abnormal sample image corresponding to the NG image has edge defects. After passing through the encoding network, the multi-scale OK features (i.e., normal sample image features) and NG features (i.e., abnormal sample image features) are extracted respectively. Figure 4 The three branches of the encoding network are shown in Figure 1. The OK features output by each branch are represented as OK feature 1, OK feature 2, and OK feature 3, respectively. The NG features output by each branch are represented as NG feature 1, NG feature 2, and NG feature 3, respectively. The NG feature 3 is input to the feature reconstruction network, and the NG feature 3 is reconstructed by the feature reconstruction network to obtain the reconstructed image features corresponding to the NG feature 3. Figure 4 Figure 2 shows the two branches of the feature reconstruction network, which output reconstructed OK feature 1 and reconstructed OK feature 2, respectively. Reconstructed OK feature 1 is then fused with NG feature 1, output by the encoding network, to produce fused image feature 1. Reconstructed OK feature 2 is fused with NG feature 2, output by the encoding network, to produce fused image feature 2. NG feature 3, fused image feature 1, and fused image feature 2 are input to the decoding network, which determines the abnormality probability score map corresponding to the second edge region of interest image.

[0093] The target loss function is determined by the first loss function and the second loss function. The first loss function is calculated based on the first difference between the reconstructed OK feature 1 and the OK feature 1 output by the encoding network, and the first difference between the reconstructed OK feature 2 and the OK feature 2 output by the encoding network. The second loss function is calculated based on the abnormal probability score map output by the decoding network and the label information on the NG map. Optionally, the target loss function Loss total The calculation method can be expressed as the following formula:

[0094]

[0095] In the above formula, represents OK feature 1 output by the encoding network (i.e., normal sample image feature), represents reconstructed OK feature 1 output by the feature reconstruction network. represents OK feature 2 output by the encoding network (i.e., normal sample image feature), represents reconstructed OK feature 2 output by the feature reconstruction network. l2 represents the first loss function. ng represents the abnormal probability score map output by the decoding network. ng represents the label information corresponding to the NG image. focal represents the second loss function. λ represents the weight corresponding to the first loss function, which is a decimal between 0 and 1. OK feature 1 and OK feature 2 correspond to different feature scales, and reconstructed OK feature 1 and reconstructed OK feature 2 correspond to different feature scales.

[0096] Then, the model parameters of the anomaly detection model are adjusted according to the target loss function. The parameters of the encoding network are fixed and do not change, and only the parameters of the other networks except the encoding network need to be adjusted. The anomaly detection model is obtained by iterative training. The termination condition of the iterative training can be any one of the following: the number of iterations reaches a preset number, or the value of the target loss function is less than or equal to a preset loss value.

[0097] In this embodiment, in the process of training the anomaly detection model, the feature reconstruction network reconstructs not the normal sample image but the normal sample image feature. Since the reconstruction of the image feature is easier than the reconstruction of the image, the amount of calculation in the model training process can be reduced, and the efficiency of the model training is improved. In addition, when calculating the abnormal probability score map, it is calculated directly according to the abnormal sample image feature and the fused image feature, that is, the fused image feature is directly decrypted, without the need for an encoding process, which greatly improves the speed of model training.

[0098] Figure 5 is a schematic flowchart of an unsupervised industrial image edge defect detection method according to another embodiment of the present application, as Figure 5 shown, the method comprises:

[0099] S502, edge detection is performed on the target image to obtain a target edge region of interest image corresponding to the target image.

[0100] The target image can be edge detected by an abnormal image edge detection algorithm. The abnormal image edge detection algorithm can be an edge region of interest detection algorithm. The detection principle of the edge region of interest detection algorithm includes, for example, Figure 2The steps shown are not repeated here.

[0101] Optionally, after edge detection is performed on the target image, an edge region corresponding to the target image is obtained. Further, by cropping the detected edge region from the target image, a target edge region of interest region image corresponding to the target image can be obtained. That is, other region images not belonging to the edge region are cropped from the target image.

[0102] S504, input the target edge region of interest region image into the pre-trained anomaly detection model, perform anomaly detection on the target edge region of interest region image through the anomaly detection model, and obtain an anomaly probability score map corresponding to the target edge region of interest region image.

[0103] The anomaly detection model is trained according to the unsupervised industrial image edge defect detection method in any of the above embodiments. The anomaly probability score map can include an anomaly probability corresponding to each image pixel included in the target edge region of interest region image.

[0104] S506, according to the anomaly probability score map, determine the anomaly information of the target image.

[0105] The technical scheme of the embodiment of the present application is adopted. By inputting the target edge region of interest region image into the pre-trained anomaly detection model, performing anomaly detection on the target edge region of interest region image through the anomaly detection model, obtaining an anomaly probability score map corresponding to the target edge region of interest region image, and further determining the anomaly information of the target image according to the anomaly probability score map. Since the target edge region of interest region image is obtained by pre-edge detection from the target image, that is, the image region where the abnormal phenomenon (i.e. edge defect) may occur is directly located from the target image, so that the model detection process does not need to process other region images except the target edge region of interest region image, which greatly improves the efficiency and accuracy of the model anomaly detection.

[0106] In one embodiment, the anomaly detection model includes an encoding network, a feature reconstruction network and a decoding network. When the anomaly detection model is used to perform anomaly detection on the target edge region of interest region image to obtain an anomaly probability score map corresponding to the target edge region of interest region image, the following mode can be executed:

[0107] The encoding network is used to extract features of the target edge region of interest region image to obtain multi-scale target image features corresponding to the target edge region of interest region image.

[0108] The feature reconstruction network is used to reconstruct features of the target image to obtain reconstructed image features corresponding to the target image features; and the reconstructed image features and the target image features are fused to obtain fused image features corresponding to the target edge region of interest region image.

[0109] By decoding the network, the abnormal probability score map corresponding to the target edge region of interest image is determined according to the target image features and the fusion image features.

[0110] In an embodiment, the target edge region of interest image includes a plurality of image pixels, and the abnormal probability score map includes an abnormal probability corresponding to each image pixel respectively. When determining the abnormal information of the target image according to the abnormal probability score map, the following steps can be performed:

[0111] Firstly, Gaussian smoothing is performed on the abnormal probability score map to obtain a smoothed probability score map.

[0112] Secondly, each abnormal probability in the smoothed probability score map is binarized to obtain a binarized abnormal probability score map.

[0113] Thirdly, the abnormal information of the target image is determined according to the binarized abnormal probability score map; the abnormal information includes at least one of the following: whether the target image is an abnormal image, and the defect position of the edge defect when the target image is an abnormal image.

[0114] The binarization process is to convert the value of each abnormal probability to 0 or 1. Optionally, for an abnormal probability greater than or equal to a preset threshold, the value of the abnormal probability is converted to 1, indicating that the image pixel at the corresponding position belongs to the abnormal region of the target image. For an abnormal probability less than the preset threshold, the value of the abnormal probability is converted to 0, indicating that the image pixel at the corresponding position does not belong to the abnormal region of the target image.

[0115] Figure 6 is a schematic diagram of an application process of an anomaly detection model provided by an embodiment of the present application. As shown in Figure 6 After the target image passes through the encoding network, multi-scale target image features are extracted. The multi-scale target image features refer to features extracted from different scales (i.e., resolutions) for the target image. There are multiple network levels in the encoding network, and the resolutions corresponding to different network levels are different. Generally, the lower the network level, the higher the corresponding resolution. By performing feature extraction on the target image through each network level in the encoding network, multi-scale (i.e., multi-resolution) target image features can be obtained. Figure 6 The three branches of the encoding network are shown schematically in the figure, and each branch corresponds to a network level, i.e., the outputs of the three branches are image features obtained by performing feature extraction on the target image through the respective corresponding network levels. The target image features output by each branch are denoted as image feature 1, image feature 2, and image feature 3, respectively. Image feature 3 is input to the feature reconstruction network, and the feature reconstruction network is used to perform feature reconstruction on image feature 3 to obtain a reconstructed image feature corresponding to image feature 3,Figure 6 Two branches of the feature reconstruction network are shown in the middle, which respectively output reconstructed feature 1 and reconstructed feature 2. Then, the reconstructed feature 1 and the image feature 1 output by the encoding network are fused to obtain the fused image feature 1. The reconstructed feature 2 and the image feature 2 output by the encoding network are fused to obtain the fused image feature 2. The image feature 3, the fused image feature 1 and the fused image feature 2 are input into the decoding network, and the abnormal probability score map corresponding to the target edge region of interest image is determined through the decoding network. Then, the abnormal probability score map is subjected to subsequent processing such as Gaussian smoothing and binarization processing, so as to determine the abnormal information of the target image according to the processed abnormal probability score map.

[0116] The unsupervised industrial image edge defect detection method provided by the above embodiment can be applied to the field of industrial product quality inspection. Taking the edge defect detection service of an industrial glass panel as an example, first, an imaging image of the glass panel is acquired by using a camera, an edge detection algorithm is used to detect the edges of the imaging image, and the detected edge region is cropped, so that a target edge region of interest image of the glass panel is obtained. The target edge region of interest image is input into a pre-trained anomaly detection model, and the target edge region of interest image of the glass panel is subjected to anomaly detection by the anomaly detection model, so that an abnormal probability score map corresponding to the target edge region of interest image of the glass panel is obtained. The abnormal probability score map includes an abnormal probability corresponding to each image pixel in the target edge region of interest image. Then, the abnormal probability score map is subjected to Gaussian smoothing and binarization processing to obtain a binarized abnormal probability score map. According to the binarized abnormal probability score map, the position of the abnormal region in the imaging image can be determined, so that whether the glass panel has an edge defect and the defect position can be determined.

[0117] In summary, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0118] The above is an unsupervised industrial image edge defect detection method provided by an embodiment of the present application. Based on the same idea, the present application also provides an unsupervised industrial image edge defect detection device.

[0119] Figure 7 is a schematic block diagram of an unsupervised industrial image edge defect detection device according to an embodiment of the present application, as shown in Figure 7 The device comprises:

[0120] The first obtaining module 71 is configured to obtain a normal sample image set; the normal sample image set includes a plurality of normal sample images.

[0121] The second obtaining module 72 is configured to obtain a mask texture image, and perform fusion processing on the mask texture image and the normal sample image to obtain an abnormal sample image set; the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample images have edge defects.

[0122] The first detection module 73 is configured to perform edge detection on the normal sample image and the abnormal sample image respectively to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image.

[0123] The model training module 74 is configured to input the first edge region of interest image and the second edge region of interest image into an abnormal detection model to be trained to perform model training, and obtain the abnormal detection model.

[0124] In one embodiment, the first obtaining module 71 performs the following steps when obtaining the normal sample image set:

[0125] Obtain a candidate image set, and the candidate image set includes N candidate normal images; N is an integer greater than 1;

[0126] Extract image features of the N candidate normal images;

[0127] According to the image features, select M normal images from the candidate image set as the normal sample images to obtain a normal sample image set; the normal sample image set includes M normal sample images; M is an integer greater than 1 and less than N.

[0128] In one embodiment, the first obtaining module 71 performs the following steps when extracting the image features of the N candidate normal images:

[0129] For each normal image, a pre-trained feature extraction model is used, and the normal image is feature-extracted according to a preset feature vector dimension to obtain image features of the normal image conforming to the feature vector dimension.

[0130] In one embodiment, the first obtaining module 71 performs the following steps when selecting M normal images from the candidate image set according to the image features to obtain a normal sample image set:

[0131] selecting any normal image in the N candidate normal images as the normal sample image, and updating the candidate image set and the normal sample image set; the updated candidate image set does not include the any normal image;

[0132] repeating the following steps until the normal sample image set includes M normal sample images:

[0133] According to the image features, determining, from the updated candidate image set, a normal image with the largest image feature distance between the normal sample image set as the normal sample image, and updating the normal sample image set.

[0134] In one embodiment, when the second acquisition module 72 performs fusion processing on the mask texture image and the normal sample image, the following steps are performed:

[0135] The mask texture image and the normal sample image are fused according to the following formula:

[0136] I ng = a * M * I texture + (1-a) * M * I ok + (1-M) * I ok

[0137] wherein I ng represents the abnormal sample image obtained after fusion, M * I texture represents the mask texture image, I texture represents the texture image provided by the texture dataset, M represents the two-dimensional mask generated by the Perlin noise generation algorithm, a represents the intensity of the texture image in the color space, I ok represents the normal sample image.

[0138] In one embodiment, the trained anomaly detection model includes an encoding network, a feature reconstruction network, a decoding network, and a loss calculation layer.

[0139] When the model training module 74 performs model training by inputting the first edge region of interest image and the second edge region of interest image into the trained anomaly detection model, the following steps are performed:

[0140] Through the encoding network, the first edge region of interest image and the second edge region of interest image are feature extracted to obtain the normal sample image features corresponding to the first edge region of interest image and the abnormal sample image features corresponding to the second edge region of interest image.

[0141] The feature reconstruction network is used for feature reconstruction of the abnormal sample image feature, to obtain a reconstructed image feature corresponding to the abnormal sample image feature; the reconstructed image feature and the abnormal sample image feature are fused to obtain a fused image feature;

[0142] The decoding network is used for determining an abnormal probability score map corresponding to the second edge region of interest image according to the abnormal sample image feature and the fused image feature; the abnormal probability score map includes an abnormal probability corresponding to each image pixel in the second edge region of interest image;

[0143] The loss calculation layer is used for calculating a target loss function of the trained abnormal detection model according to the normal sample image feature, the reconstructed image feature, the abnormal probability score map and the label information; and the trained abnormal detection model is adjusted in parameters according to the target loss function.

[0144] In one embodiment, the model training module 74 performs the following steps when calculating a target loss function of the trained abnormal detection model according to the normal sample image feature, the reconstructed image feature, the abnormal probability score map and the label information:

[0145] determining a first loss function of the trained abnormal detection model according to a first difference between the normal sample image feature and the reconstructed image feature;

[0146] determining a second loss function of the trained abnormal detection model according to a second difference between the abnormal probability score map and the label information;

[0147] determining the target loss function according to the first loss function and the second loss function.

[0148] In one embodiment, the model training module 74 performs the following steps when determining the target loss function according to the first loss function and the second loss function:

[0149] The target loss function is determined according to the following formula:

[0150]

[0151] wherein, Loss total represents the target loss function, and represents a normal sample image feature of different scales, and represents a reconstructed image feature of different scales, Loss l2 represents the first loss function, Sng representing the anomaly probability score map, T ng representing the label information corresponding to the abnormal sample image, Loss focal representing the second loss function, λ represents the weight corresponding to the first loss function.

[0152] The device according to the embodiment of the present application, by acquiring a normal sample image set, the normal sample image set including a plurality of normal sample images, and acquiring a mask texture image, fusing the mask texture image and the normal sample image to obtain an abnormal sample image set, the abnormal sample image set including a plurality of abnormal sample images (i.e. images with edge defects) and label information corresponding to the abnormal sample images; performing edge detection on the normal sample images and the abnormal sample images respectively to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images; and then inputting the first edge region of interest image and the second edge region of interest image into an abnormality detection model to be trained to train the model and obtain the abnormality detection model. It can be seen that, when training the abnormality detection model, the model is not trained directly according to the complete sample image, but is trained according to the edge region of interest images (including the first edge region of interest image and the second edge region of interest image) corresponding to the normal sample images and the abnormal sample images respectively. Since the complete sample image includes various image regions irrelevant to abnormality detection, such as background regions, other regions that do not need to be subjected to abnormality detection, etc., if the complete sample image is directly subjected to abnormality detection, the time cost and the calculation cost of model training will be significantly increased. Therefore, by locating the edge region of interest image in the sample image, the model training process does not need to process other regions of the image other than the edge region of interest image, which greatly improves the model training efficiency. In addition, by locating the edge region of interest image in advance, which is more likely to have abnormalities, the accuracy of abnormality detection in the model training process can be improved, thereby improving the model performance.

[0153] Figure 8 is a schematic block diagram of an unsupervised industrial image edge defect detection device according to an embodiment of the present application, as shown in Figure 8 the device comprises:

[0154] a second detection module 81, configured to perform edge detection on a target image to obtain a target edge region of interest image corresponding to the target image;

[0155] The anomaly detection module 82 is configured to input the target edge region of interest image into a pre-trained anomaly detection model, perform anomaly detection on the target edge region of interest image by using the anomaly detection model, and obtain an anomaly probability score map corresponding to the target edge region of interest image; the anomaly detection model is trained according to the unsupervised industrial image edge defect detection method in any one of the above embodiments.

[0156] The determination module 83 is configured to determine anomaly information of the target image according to the anomaly probability score map.

[0157] In one embodiment, the anomaly detection model comprises an encoding network, a feature reconstruction network and a decoding network.

[0158] When the anomaly detection module 82 performs anomaly detection on the target edge region of interest image by using the anomaly detection model to obtain an anomaly probability score map corresponding to the target edge region of interest image, the following steps are performed:

[0159] The encoding network is used to perform feature extraction on the target edge region of interest image to obtain a multi-scale target image feature corresponding to the target edge region of interest image.

[0160] The feature reconstruction network is used to perform feature reconstruction on the target image feature to obtain a reconstructed image feature corresponding to the target image feature; the reconstructed image feature and the target image feature are fused to obtain a fused image feature corresponding to the target edge region of interest image.

[0161] The decoding network is used to determine the anomaly probability score map corresponding to the target edge region of interest image according to the target image feature and the fused image feature.

[0162] In one embodiment, the target edge region of interest image comprises a plurality of image pixels; and the anomaly probability edge region of interest comprises an anomaly probability corresponding to each image pixel.

[0163] When the determination module 83 determines the anomaly information of the target image according to the anomaly probability score map, the following steps are performed:

[0164] The anomaly probability score map is subjected to Gaussian smoothing processing to obtain a smoothed probability score map.

[0165] Each anomaly probability in the smoothed probability score map is subjected to binarization processing to obtain a binarized anomaly probability score map.

[0166] According to the binarized anomaly probability score map, determine the anomaly information of the target image; the anomaly information includes at least one of the following: whether the target image is an abnormal image, and a defect position of an edge defect when the target image is an abnormal image.

[0167] By inputting the target edge region of interest image into the pre-trained anomaly detection model, the device of the embodiment of the present application performs anomaly detection on the target edge region of interest image through the anomaly detection model to obtain an anomaly probability score map corresponding to the target edge region of interest image, and then determines the anomaly information of the target image according to the anomaly probability score map. Since the target edge region of interest image is obtained by performing edge detection on the target image in advance, that is, the image region where the abnormal phenomenon (i.e., edge defect) may occur has been directly located from the target image, so that the model detection process does not need to process other region images except the target edge region of interest image, which greatly improves the efficiency and accuracy of the model anomaly detection.

[0168] Those skilled in the art should understand that, Figure 7 and Figure 8 The unsupervised industrial image edge defect detection device in the above embodiments can be used to implement the unsupervised industrial image edge defect detection method described above, and the details of the description should be similar to the description of the method part above. To avoid tediousness, this will not be described again.

[0169] Based on the same idea, the present application also provides an electronic device, as shown in Figure 9 The electronic device can have a large difference due to different configurations or performances, and can include one or more processors 901 and memories 902, and the memories 902 can store one or more storage applications or data. The memory 902 can be temporary storage or persistent storage. The applications stored in the memory 902 can include one or more modules (not shown in the figure), and each module can include a series of computer executable instructions in the electronic device. Further, the processor 901 can be configured to communicate with the memory 902 and execute a series of computer executable instructions in the memory 902 on the electronic device. The electronic device can also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input / output interfaces 905, and one or more keyboards 906.

[0170] In particular embodiments, an electronic device includes memory, and one or more programs, wherein one or more programs are stored in the memory and accessible by one or more processors for one or more programs include one or more modules, and each module can include a set of computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform a computer process. For example, a set of computer-executable instructions can form a module that, when executed by the one or more processors, cause the electronic device to perform a computer process.

[0171] obtain a normal sample image set; the normal sample image set includes a plurality of normal sample images;

[0172] obtain a mask texture image, and generate an abnormal sample image set according to the mask texture image; the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample images have edge defects;

[0173] respectively perform edge detection on the normal sample images and the abnormal sample images to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images;

[0174] input the first edge region of interest image and the second edge region of interest image into an abnormal detection model to be trained to perform model training, and obtain the abnormal detection model.

[0175] The technical solutions of the embodiments of the present application are used in training the abnormal detection model. Instead of directly training according to complete sample images, the training is performed according to the edge region of interest images (including the first edge region of interest image and the second edge region of interest image) corresponding to the normal sample images and the abnormal sample images. Since the complete sample images include various image regions irrelevant to abnormal detection, such as background regions, other regions that do not need to be subjected to abnormal detection, etc., if the complete sample material images are directly subjected to abnormal detection, the time cost and the calculation cost of model training will be significantly increased. Therefore, by locating the edge region of interest images in the sample images, the other region images outside the edge region of interest images do not need to be processed in the model training process, which greatly improves the model training efficiency. In addition, by locating the edge region of interest images that are more likely to have abnormalities in advance, the accuracy of abnormal detection in the model training process can be improved, thereby improving the model performance.

[0176] In another embodiment, the electronic device includes a memory and one or more programs, wherein one or more programs are stored in the memory and the one or more programs can include one or more modules, and each module can include a series of computer-executable instructions in the electronic device and is configured to be executed by one or more processors to perform the following computer-executable instructions:

[0177] performing edge detection on the target image to obtain a target edge region of interest image corresponding to the target image;

[0178] inputting the target edge region of interest image into a pre-trained anomaly detection model, performing anomaly detection on the target edge region of interest image by the anomaly detection model to obtain an anomaly probability score map corresponding to the target edge region of interest image; the anomaly detection model is trained according to the unsupervised industrial image edge defect detection method in any embodiment;

[0179] determining anomaly information of the target image according to the anomaly probability score map.

[0180] The technical scheme of the embodiment of the present application is adopted, the target edge region of interest image is input into a pre-trained anomaly detection model, the anomaly detection model is used to perform anomaly detection on the target edge region of interest image, an anomaly probability score map corresponding to the target edge region of interest image is obtained, and then the anomaly information of the target image is determined according to the anomaly probability score map. Since the target edge region of interest image is obtained by performing edge detection on the target image in advance, that is, the image region where the abnormal phenomenon (i.e., edge defect) may occur is directly located from the target image, so that the model detection process does not need to process other region images except the target edge region of interest image, and the efficiency and accuracy of the model anomaly detection are greatly improved.

[0181] The embodiment of the present application also provides a computer readable storage medium, which stores one or more computer programs, the one or more computer programs include instructions, and when the instructions are executed by an electronic device including a plurality of application programs, the electronic device can execute each process of the anomaly detection method embodiment and is specifically used to execute:

[0182] obtaining a normal sample image set; the normal sample image set includes a plurality of normal sample images;

[0183] obtaining a mask texture image, and generating an abnormal sample image set according to the mask texture image; the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample images have edge defects;

[0184] performing edge detection on the normal sample images and the abnormal sample images respectively to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images;

[0185] inputting the first edge region of interest image and the second edge region of interest image into an abnormal detection model to be trained to perform model training, and obtaining the abnormal detection model.

[0186] When training the abnormal detection model, the technical solution of the embodiment of the present application is not directly trained according to the complete sample image, but is trained according to the edge region of interest images (including the first edge region of interest image and the second edge region of interest image) corresponding to the normal sample images and the abnormal sample images respectively. Since the complete sample image includes various image regions irrelevant to abnormal detection, such as background regions and other regions that do not need to be subjected to abnormal detection, if the complete sample material image is directly subjected to abnormal detection, the time cost and the calculation cost of model training will be significantly increased. Therefore, by locating the edge region of interest image in the sample image, the other region images outside the edge region of interest image do not need to be processed in the model training process, which greatly improves the model training efficiency. In addition, by pre-locating the edge region of interest image where the abnormality is more likely to exist, the accuracy of abnormal detection in the model training process can be improved, thereby improving the model performance.

[0187] The embodiment of the present application also proposes a computer readable storage medium storing one or more computer programs, the one or more computer programs including instructions that, when executed by an electronic device including a plurality of application programs, can enable the electronic device to perform each process of the above-mentioned abnormal detection method embodiment, and are specifically used to perform:

[0188] performing edge detection on a target image to obtain a target edge region of interest image corresponding to the target image;

[0189] inputting the target edge region of interest image into a pre-trained abnormal detection model, performing abnormal detection on the target edge region of interest image through the abnormal detection model to obtain an abnormal probability score map corresponding to the target edge region of interest image; the abnormal detection model is trained according to the unsupervised industrial image edge defect detection method in any embodiment;

[0190] According to the anomaly probability score map, anomaly information of the target image is determined.

[0191] By inputting the target edge region of interest image into the pre-trained anomaly detection model, the anomaly detection model is used to perform anomaly detection on the target edge region of interest image, and an anomaly probability score map corresponding to the target edge region of interest image is obtained. Then, according to the anomaly probability score map, the anomaly information of the target image is determined. Since the target edge region of interest image is obtained by performing edge detection on the target image in advance, that is, the image region where the abnormal phenomenon (i.e., edge defect) may occur is directly located from the target image, so that the model detection process does not need to process other region images except the target edge region of interest image, which greatly improves the efficiency and accuracy of the model anomaly detection.

[0192] The system, device, module or unit described in the above embodiments can be implemented by a computer or an entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0193] For the convenience of description, the above device is described as various units by function. Of course, the functions of each unit can be implemented in the same or more software and / or hardware in the implementation of the present application.

[0194] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0195] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the computer or other programmable data processing apparatus produce a device that implements the functions described in the flowcharts and / or block diagrams.Figure 1 one or more processes and / or blocks Figure 1 means for performing the function specified by the block or blocks.

[0196] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more processes and / or blocks Figure 1 means for performing the function specified by the block or blocks.

[0197] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more processes and / or blocks Figure 1 Figure 1 means for performing the function specified by the block or blocks.

[0198] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0199] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory. The memory can also include non-volatile memory, such as read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or non-volatile random access memory (NVRAM). The memory is an example of computer readable media.

[0200] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0201] It is also to be noted that the terms "comprising", "including", and "having" or variations thereof herein, are intended to be open-ended terms that specify the presence of the stated elements but do not preclude the presence of additional elements. It is also to be noted that the term "coupled" is intended to be an open term that includes two elements that are either directly connected to each other or that are connected to each other through one or more intermediate elements. In contrast, the term "connected" is intended to be a closed term that includes two elements that are either directly connected to each other or that are connected to each other through one or more intermediate elements.

[0202] The present application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0203] The various embodiments in this application are described in progressive manner, and the same or similar parts between embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the system embodiments are described in a relatively simple manner because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0204] The above only describes the embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. An unsupervised industrial image edge defect detection method, characterized in that, The method comprises the following steps: obtaining a normal sample image set; the normal sample image set comprises a plurality of normal sample images; obtaining a mask texture image, fusing the mask texture image and the normal sample image to obtain an abnormal sample image set; the abnormal sample image set comprises a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample image has an edge defect; respectively performing edge detection on the normal sample image and the abnormal sample image to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image; inputting the first edge region of interest image and the second edge region of interest image into a trained abnormal detection model to perform model training, and obtaining the abnormal detection model.

2. The method of claim 1, wherein, The method comprises the following steps: obtaining a normal sample image set; the normal sample image set comprises a plurality of normal sample images; obtaining a mask texture image, fusing the mask texture image and the normal sample image to obtain an abnormal sample image set; the abnormal sample image set comprises a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample image has an edge defect; 3. The method of claim 2, wherein, respectively performing edge detection on the normal sample image and the abnormal sample image to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image; inputting the first edge region of interest image and the second edge region of interest image into a trained abnormal detection model to perform model training, and obtaining the abnormal detection model.

4. The method of claim 2, wherein, The method comprises the following steps: obtaining a normal sample image set; the normal sample image set comprises a plurality of normal sample images; obtaining a mask texture image, fusing the mask texture image and the normal sample image to obtain an abnormal sample image set; the abnormal sample image set comprises a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample image has an edge defect; 5. The method of claim 1, wherein, respectively performing edge detection on the normal sample image and the abnormal sample image to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image; inputting the first edge region of interest image and the second edge region of interest image into a trained abnormal detection model to perform model training, and obtaining the abnormal detection model. I ng = a * M * I texture + (1 - a) * M * I ok + (1 - M) * I ok wherein I ng represents the abnormal sample image obtained after fusion, M*I texture represents the mask texture image, I texture represents the texture image provided by the texture dataset, M represents a two-dimensional mask generated by a Perlin noise generation algorithm, and a represents the intensity of the texture image in the color space, I ok represents the normal sample image.

6. The method of claim 1, wherein, The method comprises the following steps: obtaining a normal sample image set; the normal sample image set comprises a plurality of normal sample images; obtaining a mask texture image, fusing the mask texture image and the normal sample image to obtain an abnormal sample image set; the abnormal sample image set comprises a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample image has an edge defect; respectively performing edge detection on the normal sample image and the abnormal sample image to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image; inputting the first edge region of interest image and the second edge region of interest image into a trained abnormal detection model to perform model training, and obtaining the abnormal detection model. The method comprises the following steps: obtaining a normal sample image set; the normal sample image set comprises a plurality of normal sample images; obtaining a mask texture image, fusing the mask texture image and the normal sample image to obtain an abnormal sample image set; the abnormal sample image set comprises a plurality of abnormal sample images and label information corresponding to the abnormal sample images; the label information is used to represent that the abnormal sample image has an edge defect; respectively performing edge detection on the normal sample image and the abnormal sample image to obtain a first edge region of interest image corresponding to the normal sample image and a second edge region of interest image corresponding to the abnormal sample image; inputting the first edge region of interest image and the second edge region of interest image into a trained abnormal detection model to perform model training, and obtaining the abnormal detection model. The first edge region of interest image and the second edge region of interest image are subjected to feature extraction through the coding network, so as to obtain normal sample image features of multiple scales corresponding to the first edge region of interest image and abnormal sample image features of multiple scales corresponding to the second edge region of interest image; The abnormal sample image features are subjected to feature reconstruction through the feature reconstruction network, so as to obtain reconstructed image features corresponding to the abnormal sample image features; the reconstructed image features and the abnormal sample image features are subjected to fusion processing, so as to obtain fused image features; The abnormal probability score map corresponding to the second edge region of interest image is determined according to the abnormal sample image features and the fused image features through the decoding network; the abnormal probability score map includes an abnormal probability corresponding to each image pixel in the second edge region of interest image; The target loss function of the trained abnormal detection model is calculated according to the normal sample image features, the reconstructed image features, the abnormal probability score map and the label information through the loss calculation layer; and the trained abnormal detection model is subjected to parameter adjustment according to the target loss function.

7. The method of claim 6, wherein, The target loss function of the trained abnormal detection model is calculated according to the normal sample image features, the reconstructed image features, the abnormal probability score map and the label information, including: A first loss function of the trained abnormal detection model is determined according to a first difference degree between the normal sample image features and the reconstructed image features; A second loss function of the trained abnormal detection model is determined according to a second difference degree between the abnormal probability score map and the label information; The target loss function is determined according to the first loss function and the second loss function.

8. The method of claim 7, wherein, The target loss function is determined according to the first loss function and the second loss function, including: The target loss function is determined according to the following formula: wherein Loss total denotes the target loss function, and denotes the normal sample image features of different scales, and denotes the reconstructed image features of different scales, Loss l2 denotes the first loss function, S ng denotes the anomaly probability score map, T ng denotes the label information corresponding to the abnormal sample image, Loss focal denotes the second loss function, and λ denotes the weight corresponding to the first loss function.

9. An unsupervised industrial image edge defect detection method, characterized in that, including: An edge detection is performed on a target image, so as to obtain a target edge region of interest image corresponding to the target image; The target edge region of interest image is input into a pre-trained abnormal detection model, and the target edge region of interest image is subjected to abnormal detection through the abnormal detection model, so as to obtain an abnormal probability score map corresponding to the target edge region of interest image; The abnormal detection model is trained according to the unsupervised industrial image edge defect detection method in any one of claims 1 to 8; Abnormal information of the target image is determined according to the abnormal probability score map.

10. The method of claim 9, wherein, The abnormal detection model includes a coding network, a feature reconstruction network and a decoding network. The target edge region of interest image is subjected to abnormal detection through the abnormal detection model, so as to obtain an abnormal probability score map corresponding to the target edge region of interest image, including: The target edge region of interest image is subjected to feature extraction through the coding network, so as to obtain target image features of multiple scales corresponding to the target edge region of interest image; The feature reconstruction network is used for feature reconstruction of the target image feature to obtain a reconstructed image feature corresponding to the target image feature; and the reconstructed image feature and the target image feature are fused to obtain a fusion image feature corresponding to the target edge region of interest image; The decoding network is used for determining the abnormal probability score map corresponding to the target edge region of interest image according to the target image feature and the fusion image feature.

11. The method of claim 9, wherein, The target edge region of interest image includes a plurality of image pixels; and the abnormal probability score map includes an abnormal probability corresponding to each of the image pixels. The determination of the abnormal information of the target image according to the abnormal probability score map includes: The abnormal probability score map is subjected to Gaussian smoothing processing to obtain a smoothed probability score map; Each abnormal probability in the smoothed probability score map is subjected to binaryzation processing to obtain a binaryzation abnormal probability score map; The abnormal information of the target image is determined according to the binaryzation abnormal probability score map; and the abnormal information includes at least one of the following: whether the target image is an abnormal image, and a defect position of an edge defect when the target image is an abnormal image.

12. An unsupervised industrial image edge defect detection apparatus, characterized by, The method includes: A first acquisition module is configured to acquire a normal sample image set; The normal sample image set includes a plurality of normal sample images; A second acquisition module is configured to acquire a mask texture image and generate an abnormal sample image set according to the mask texture image; the abnormal sample image set includes a plurality of abnormal sample images and label information corresponding to the abnormal sample images; and the label information is used to represent that the abnormal sample images have edge defects. A first detection module is configured to perform edge detection on the normal sample images and the abnormal sample images respectively to obtain a first edge region of interest image corresponding to the normal sample images and a second edge region of interest image corresponding to the abnormal sample images. A model training module is configured to input the first edge region of interest image and the second edge region of interest image into an abnormal detection model to be trained for model training, and obtain the abnormal detection model.

13. An unsupervised industrial image edge defect detection device, characterized in that: The method includes: A second detection module is configured to perform edge detection on a target image to obtain a target edge region of interest image corresponding to the target image; An abnormal detection module is configured to input the target edge region of interest image into a pre-trained abnormal detection model, perform abnormal detection on the target edge region of interest image through the abnormal detection model, and obtain an abnormal probability score map corresponding to the target edge region of interest image; The abnormal detection model is trained according to the unsupervised industrial image edge defect detection method of any one of claims 1 to 8; A determination module is configured to determine abnormal information of the target image according to the abnormal probability score map.

14. A computer-readable storage medium, characterized in that, The storage medium is configured to store a computer program, which can be executed by a processor to implement the unsupervised industrial image edge defect detection method of any one of claims 1 to 11.

Images