Industrial image anomaly detection method and system based on feature-level anomaly synthesis and reconstruction

Abstract

The application discloses an industrial image anomaly detection method and system based on feature-level abnormal synthesis and reconstruction, and the method is as follows: S1, acquiring an image, and extracting multi-scale local features through a pre-trained network; S2, adjusting the features of different scales to a unified size, connecting the features of the unified size, and obtaining a feature map; S3, transferring the feature map of S2 to a target domain through feature adaptation, and obtaining a new feature map; S4, adding noise to the feature map obtained in S3, and obtaining a feature map simulating an anomaly; S5, inputting the feature map obtained in S4 into a feature reconstruction network, and reconstructing the features into features without anomalies; S6, calculating a loss through the difference between the feature map before adding noise and the reconstructed feature map, training the feature reconstruction network according to the loss value; and S7, reconstructing the features by using the trained network, and obtaining a segmentation map according to the difference before and after the feature reconstruction. The application solves the problems of high calculation cost, unreal synthesized anomaly and reconstructed anomaly area of the existing anomaly detection method.

Description

Technical Field

[0001] This invention belongs to the field of anomaly detection technology in artificial intelligence, specifically relating to a high-accuracy, high-speed industrial image anomaly detection method and system based on feature-level anomaly synthesis and reconstruction. Background Technology

[0002] In industrial applications, anomaly detection is a crucial tool for ensuring production continuity and improving efficiency. By monitoring production equipment in real time, it provides early warnings of potential faults, enabling companies to take preventative measures before problems escalate and reducing maintenance costs. Furthermore, anomaly detection is used to optimize production lines by monitoring process parameters, ensuring product quality meets standards, identifying production bottlenecks, and ultimately increasing output. With the rapid development of the manufacturing industry, the importance of industrial product quality control has become increasingly prominent. Against this backdrop, traditional product quality inspection methods have gradually revealed their limitations. In recent years, breakthroughs in computer vision and deep learning technologies have provided powerful tools for industrial surface quality inspection. Computer vision-based inspection systems can run in real time on the production line, providing immediate feedback on inspection results. However, due to the lack of defect data and the difficulty of annotation, unsupervised methods are often used in industrial applications. Unsupervised methods use only normal samples for training during the training phase and are currently mainly divided into three categories: embedding-based, synthesis-based, and reconstruction-based methods. Embedding-based methods suffer from high computational complexity or memory consumption; synthesis-based methods synthesize images that are not realistic enough and differ from real anomaly images; and reconstruction-based methods can reconstruct abnormal inputs well but are prone to missed detections. Summary of the Invention

[0003] To address the aforementioned problems in existing technologies, namely the high computational cost, unrealistic anomaly synthesis, and incomplete reconstruction of anomaly regions in existing anomaly detection methods, this invention provides a high-accuracy, high-speed industrial image anomaly detection method and system based on feature-level anomaly synthesis and reconstruction.

[0004] The present invention adopts the following technical solution:

[0005] An industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction, comprising the following steps:

[0006] S1. Acquire images and extract multi-scale local features using a pre-trained network;

[0007] S2. Adjust features of different scales to a uniform size, connect features of the uniform size, and obtain a feature map;

[0008] S3. Through feature adaptation, the feature map from step S2 is transferred to the target domain to obtain a new feature map;

[0009] S4. Add noise to the feature map obtained in step S3 to obtain a feature map simulating an anomaly.

[0010] S5. Input the feature map obtained in step S4 into the feature reconstruction network to reconstruct the features into features without anomalies;

[0011] S6. Calculate the loss by the difference between the feature maps before and after adding noise, and train the feature reconstruction network based on the loss value to improve the accuracy of feature reconstruction.

[0012] S7. After the network training is completed, the features from step S3 are reconstructed using the post-trained network, and the segmentation map is obtained based on the differences before and after feature reconstruction.

[0013] Preferably, step S1 is as follows: Obtain a visible light image I of the industrial production line product, and extract multi-scale local features of the image using a pre-trained network model φ. For an input image I∈R... H×W×C Where H represents the image height, W represents the image width, and C represents the number of image channels, the features z at different scales can be obtained through the network φ. j =φ j (I), where j represents the different layers of the network. To obtain multi-scale local features, adaptive average pooling is used to aggregate neighborhood features, resulting in multi-scale local features F. j =agg(z j ).

[0014] Preferably, in step S2, in order to combine features at different scales, the features at different scales are upsampled to a uniform size, i.e., the largest scale size, and these uniformly sized features are connected to obtain the feature map F. m , can be represented as follows:

[0015] F m =f cat (f resize (F f′ ,(H l W l )))|j′∈J

[0016] Among them, f cat (·) indicates a join operation, f resize (·) indicates a resizing operation, H l W represents the height at its maximum scale. l J represents the width at the maximum scale, and J represents the set of layers used in the network φ.

[0017] Preferably, in step S3, due to the feature map F obtained in the above steps mTo more closely resemble the training set used by the pre-trained network, a feature adapter is needed to transform the resulting feature map F. m The feature map is transferred to the target domain to adapt to its distribution, resulting in a new feature map F. p The feature adapter is essentially a simple fully connected layer used to transform pre-trained features into a representation more suitable for the current task. This step can be represented as:

[0018] F p =f adapter (F m )

[0019] Among them, f adapter This is for feature adaptation operations.

[0020] Preferably, step S4 is as follows: after obtaining the extracted feature map F p Subsequently, in order for the model to be able to identify anomalies, it is necessary to feed F during the training phase. p Noise is added to simulate anomalies so that the model can learn the distribution of anomalies. Specifically, an anomaly mask is first generated using Berlin noise with a certain threshold. Gaussian noise is then added to the original features based on this mask; that is, noise is added to the parts where the mask is 1, and not to the parts where it is 0. This process can be represented as follows:

[0021] F g =F p +M a ⊙G

[0022] Among them, F g M represents the feature map after adding noise. a This represents the mask generated using Berlin noise, where G represents Gaussian noise and ⊙ represents pixel-wise multiplication. This process is performed only during the training phase; during the testing phase, F is directly applied. p As F g Continue with the next steps.

[0023] Preferably, in step S5, the feature map F g The input is fed into a feature reconstruction network, which reconstructs the features with anomalies into feature maps F containing only normal regions. r Specifically, a convolutional autoencoder containing only 1×1 convolutions and a ReLU activation function are used as features to reconstruct the network.

[0024] Preferably, in step S6, the loss is calculated by the difference between the feature maps before and after adding noise, using L2 distance as the loss function, and is calculated only during the training phase, as follows:

[0025]

[0026] Where h represents the height of the feature, w represents the width of the feature, and ||·||2 represents the L2 distance between the pixels on the feature map before noise is added and the pixels on the reconstructed feature map.

[0027] Preferably, step S7 is as follows: Features are reconstructed using the trained network, and a segmentation map is obtained by analyzing the feature differences before and after reconstruction. The L2 distance is used as the calculation method, and the segmentation map S can be represented as follows:

[0028] S = ||F g -F r ||2

[0029] The resulting segmentation image S is the detection result of the present invention for anomaly detection in industrial images.

[0030] This invention also discloses an industrial image anomaly detection system based on feature-level anomaly synthesis and reconstruction, used to perform the above method, comprising the following modules:

[0031] Image acquisition and feature extraction module: Acquires images and extracts multi-scale local features through a pre-trained network;

[0032] Unified Feature Map Size Module: Adjusts features of different scales to a unified size, connects features of the unified size, and obtains a feature map;

[0033] Feature adaptation module: Through feature adaptation, the feature map is transferred to the target domain to obtain a new feature map;

[0034] Noise addition module: Adds noise to the feature map obtained by the feature adaptation module to obtain a feature map simulating anomalies;

[0035] Feature reconstruction module: Inputs the feature map obtained by the noise addition module into the feature reconstruction network to reconstruct the features into features without anomalies;

[0036] Loss calculation module: Calculates the loss by comparing the feature maps before and after adding noise, and trains the feature reconstruction network based on the loss value;

[0037] The detection result acquisition module uses the trained network to reconstruct features and obtains a segmentation map based on the differences before and after feature reconstruction.

[0038] This invention provides a high-accuracy, high-speed method and system for industrial image anomaly detection based on feature-level anomaly synthesis and reconstruction. This invention effectively improves the capability of industrial image anomaly detection. The technical solution provided by this invention can be used for intelligent quality inspection of products in industrial production lines, improving the efficiency of industrial production lines while increasing average product quality, reducing labor costs, promoting the intelligentization of industrial systems, and comprehensively reducing production costs. Attached Figure Description

[0039] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0040] Figure 1 This is a flowchart of a preferred embodiment of the present invention for a high-accuracy, high-speed industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction;

[0041] Figure 2 This is a schematic diagram showing the results of testing using a preferred embodiment of the present invention.

[0042] Figure 3 This is a block diagram of an industrial image anomaly detection system based on feature-level anomaly synthesis and reconstruction, according to a preferred embodiment of the present invention. Detailed Implementation

[0043] To make the embodiments, technical solutions, and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0044] like Figure 1 As shown, this embodiment provides a high-accuracy, high-speed industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction. The method includes the following steps:

[0045] Step S100: Obtain the visible light image I of the industrial production line product, and extract multi-scale local features of the image using a pre-trained network model φ. For an input image I∈R H×W×C Where H represents the image height, W represents the image width, and C represents the number of image channels, the features z at different scales can be obtained through the network φ. j =φ j (I), where j represents the different layers of the network. To obtain multi-scale local features, adaptive average pooling is used to aggregate neighborhood features, resulting in multi-scale local features F. j =agg(z j ).

[0046] Step S200: In order to combine features at different scales, the features at different scales are upsampled to a uniform size, i.e., the largest scale size, and these uniformly sized features are concatenated to obtain the feature map F. m , can be represented as follows:

[0047] F m =f cat (f resize(F j′ ,(H l W l )))|j′∈J

[0048] Among them, f cat (·) indicates a join operation, f resize (·) indicates a resizing operation, H l W represents the height at its maximum scale. l J represents the width at the maximum scale, and J represents the set of layers used in the network φ.

[0049] Step S300, due to the feature map F obtained in the above steps m To more closely resemble the training set used by the pre-trained network, a feature adapter is needed to transform the resulting feature map F. m The feature map is transferred to the target domain to adapt to its distribution, resulting in a new feature map F. p The feature adapter is essentially a simple fully connected layer used to transform pre-trained features into a representation more suitable for the current task. This step can be represented as:

[0050] F p =f adapter (F m )

[0051] Among them, f adapter This is for feature adaptation operations.

[0052] Step S400, after obtaining the extracted feature map F p Subsequently, in order for the model to be able to identify anomalies, it is necessary to feed F during the training phase. p Noise is added to simulate anomalies so that the model can learn the distribution of anomalies. Specifically, an anomaly mask is first generated using Berlin noise with a certain threshold. Gaussian noise is then added to the original features based on this mask; that is, noise is added to the parts where the mask is 1, and not to the parts where it is 0. This process can be represented as follows:

[0053] F g =F p +M a ⊙G

[0054] Among them, F g M represents the feature map after adding noise. a This represents the mask generated using Berlin noise, where G represents Gaussian noise and ⊙ represents pixel-wise multiplication. This process is performed only during the training phase; during the testing phase, F is directly applied. p As F g Continue with the next steps.

[0055] Step S500, transfer feature map F gThe input is fed into a feature reconstruction network, which reconstructs the abnormal features into features F that only contain normal regions. r Specifically, a convolutional autoencoder containing only 1×1 convolutions and a ReLU activation function are used as features to reconstruct the network.

[0056] Step S600: Calculate the loss by comparing the feature map before and after adding noise, using L2 distance as the loss function. This loss is calculated only during the training phase and can be represented as follows:

[0057]

[0058] Where h represents the height of the feature, w represents the width of the feature, and ||·||² represents the L2 distance between pixels in the feature map before noise is added and pixels in the reconstructed feature map. This represents the pixel at position i pixels high and j pixels wide in the feature map before noise was added. This represents the pixel in the reconstructed feature map at a height of i pixels and a width of j pixels.

[0059] Step S700: After network training is completed, the features of S300 are reconstructed using the trained network. The segmentation map is obtained based on the feature differences before and after feature reconstruction. The L2 distance is used as the calculation method. The segmentation map S can be represented as follows:

[0060] S = ||F g -F r ||2

[0061] The resulting segmentation image S is the detection result of the present invention for anomaly detection in industrial images.

[0062] refer to Figure 2 In this embodiment, the data used are images of industrial products, with manually annotated anomaly areas included for reference. By inputting the image to be detected into the network proposed in this invention, the network can detect the corresponding results for the image. Figure 2 As shown, the detection heatmap is the segmentation map S in step S700 of the above embodiment. The detection heatmap is converted into a detection binary map by setting a threshold of 0.5, where pixels above the threshold are set to 1, and pixels below the threshold are set to 0. Regions with a pixel value of 1 are defined as abnormal regions in the original image and are represented in white. As can be seen from this figure, the detection results using the network proposed in this invention are very similar to those obtained through manual annotation, and can be used in practical applications to reduce labor costs and improve production efficiency.

[0063] See attached document Figure 3 , Figure 3This is a system block diagram of a preferred embodiment of the present invention. This embodiment provides a high-accuracy, high-speed industrial image anomaly detection system based on feature-level anomaly synthesis and reconstruction, specifically including the following modules:

[0064] Image acquisition and feature extraction module: Acquires a visible light image I of an industrial production line product, and extracts multi-scale local features of the image using a pre-trained network model φ. For an input image I∈R H×W×C Where H represents the image height, W represents the image width, and C represents the number of image channels, the features z at different scales can be obtained through the network φ. j =φ j (I), where j represents the different layers of the network. To obtain multi-scale local features, adaptive average pooling is used to aggregate neighborhood features, resulting in multi-scale local features F. j =agg(z j ).

[0065] Unified Feature Map Size Module: To combine features at different scales, features at different scales are upsampled to a unified size, i.e., the largest scale size, and these unified-size features are concatenated to obtain the feature map F. m , can be represented as follows:

[0066] F m =f cat (f resize (F j′ ,(H k W l )))|j′∈J

[0067] Among them, f cat (·) indicates a join operation, f resize (·) indicates a resizing operation, H l W represents the height at its maximum scale. l J represents the width at the maximum scale, and J represents the set of layers used in the network φ.

[0068] Feature adaptation module: Due to the feature map F obtained above m To more closely resemble the training set used by the pre-trained network, a feature adapter is needed to transform the resulting feature map F. m The feature map is transferred to the target domain to adapt to its distribution, resulting in a new feature map F. p The feature adapter is essentially a simple fully connected layer used to transform pre-trained features into a representation more suitable for the current task. It can be represented as:

[0069] F p =f adapter (F m )

[0070] Among them, f adapter This is for feature adaptation operations.

[0071] Noise addition module: After obtaining the extracted feature map F p Subsequently, in order for the model to be able to identify anomalies, it is necessary to feed F during the training phase. p Noise is added to simulate anomalies so that the model can learn the distribution of anomalies. Specifically, an anomaly mask is first generated using Berlin noise with a certain threshold. Gaussian noise is then added to the original features based on this mask; that is, noise is added to the parts where the mask is 1, and not to the parts where it is 0. This process can be represented as follows:

[0072] F g =F p +M a ⊙G

[0073] Among them, F g M represents the feature map after adding noise. a This represents the mask generated using Berlin noise, where G represents Gaussian noise and ⊙ represents pixel-wise multiplication. This process is performed only during the training phase; during the testing phase, F is directly applied. p As F g Continue with the next steps.

[0074] Feature Reconstruction Module: Reconstructs feature map F g The input is fed into a feature reconstruction network, which reconstructs the features with anomalies into feature maps F containing only normal regions. r Specifically, a convolutional autoencoder containing only 1×1 convolutions and a ReLU activation function are used as features to reconstruct the network.

[0075] Loss calculation module: Calculates the loss by comparing the feature maps before and after noise addition, and trains the feature reconstruction network based on the loss value. Using L2 distance as the loss function, it is calculated only during the training phase, and can be represented as follows:

[0076]

[0077] Where h represents the height of the feature, w represents the width of the feature, and ||·||2 represents the L2 distance between the pixels on the feature map before noise is added and the pixels on the reconstructed feature map.

[0078] The detection result acquisition module: features are reconstructed using the trained network, and a segmentation map is obtained by analyzing the feature differences before and after reconstruction. The L2 distance is used as the calculation method, and the segmentation map S can be represented as follows:

[0079] S = ||F g -F r ||2

[0080] The resulting segmentation image S is the detection result of the present invention for anomaly detection in industrial images.

[0081] This invention provides a high-accuracy, high-speed method and system for industrial image anomaly detection based on feature-level anomaly synthesis and reconstruction, effectively improving the capability of industrial image anomaly detection. The method provided by this invention can be used for intelligent quality inspection of products in industrial production lines, improving the efficiency of industrial production lines while increasing average product quality, reducing labor costs, promoting the intelligentization of industrial systems, and comprehensively reducing production costs.

[0082] In summary, this invention provides a high-accuracy, high-speed industrial image anomaly detection method and system based on feature-level anomaly synthesis and reconstruction, aiming to solve the problems of high computational cost, unrealistic anomaly synthesis, and difficulty in effectively reconstructing anomaly regions in existing anomaly detection methods. This invention first acquires visible light images of industrial production line products and extracts multi-scale local features of the images through a pre-trained network model. Next, these features are upsampled to a uniform size and combined, and then a feature adapter is used to adjust the features to fit the target domain distribution. To train the model to recognize anomalies, noise is added to the processed features during the training phase to simulate anomalies. Subsequently, a convolutional autoencoder containing only 1×1 convolutions and a ReLU activation function are used as a feature reconstruction network to attempt to reconstruct features containing anomalies into features containing only normal regions. The loss is calculated by comparing the difference between the feature maps before and after noise addition, and a segmentation map for anomaly detection is generated based on the difference in the feature map before and after reconstruction. This invention can be used in intelligent quality inspection of industrial production line products, improving production efficiency and product quality while reducing labor costs and promoting the intelligent development of industrial systems.

[0083] It should be noted that in the description of this invention, the terms "input," "extraction," "training," "local," "multi-scale," "unified," "transfer," and "reconstruction," etc., are used to clearly express and understand the technical features of this invention, rather than to limit its implementation. In practical applications, relevant parameters and methods can be adjusted according to specific scenarios and needs.

[0084] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "application," "combination," "connection," etc., should be interpreted broadly. For example, it can mean directly using related methods and technologies, or modifying or extending them; it can mean tightly combining multiple methods and technologies to form a holistic solution, or selectively using a portion of them; it can mean integrating different data into a whole so that they can work collaboratively, or realizing the interaction between different data through interfaces or middleware. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0085] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to those processes, articles, or apparatus / devices.

[0086] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction, characterized in that, Includes the following steps: S1. Acquire images and extract multi-scale local features using a pre-trained network; S2. Adjust features of different scales to a uniform size, connect features of the uniform size, and obtain a feature map; S3. Through feature adaptation, the feature map from step S2 is transferred to the target domain to obtain a new feature map; S4. Add noise to the feature map obtained in step S3 to obtain a feature map simulating an anomaly. S5. Input the feature map obtained in step S4 into the feature reconstruction network to reconstruct the features into features without anomalies; S6. Calculate the loss by the difference between the feature map before adding noise and the reconstructed feature map, and train the feature reconstruction network based on the loss value; S7. Reconstruct features using the trained network and obtain a segmentation map based on the differences before and after feature reconstruction. Step S4 is as follows: An anomaly mask is generated by applying Berlin noise and setting a threshold; this mask is then used as the basis for applying it to the feature map F. p Add Gaussian noise, the specific expression is as follows: in, M represents the feature map after adding noise. a This represents a mask generated using Berlin noise, where G represents Gaussian noise. This indicates a pixel-by-pixel multiplication operation; In step S5, a convolutional autoencoder containing only 1×1 convolutions and a ReLU activation function are used as the feature reconstruction network; the feature maps are... The input is fed into a feature reconstruction network, which reconstructs the feature map with anomalies into a feature map F containing only normal regions. r ; In step S6, the loss is calculated by comparing the feature map before and after adding noise, using the L2 distance as the loss function. The specific expression is as follows: Where h represents the height of the feature, and w represents the width of the feature. This represents the L2 distance between pixels in the feature map before noise is added and pixels in the reconstructed feature map. This represents the pixel at position i pixels high and j pixels wide in the feature map before noise was added. This represents the pixel in the reconstructed feature map at a height of i pixels and a width of j pixels. Step S7 is as follows: A segmentation map is obtained by analyzing the feature differences before and after reconstruction. The L2 distance is used as the calculation method, and the segmentation map S is represented as follows: The resulting segmentation map S represents the detection result of anomaly detection in industrial images.

2. The industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction as described in claim 1, characterized in that, Step S1 is as follows: Obtain the visible light image I of the industrial production line product, and extract multi-scale local features of the image through a pre-trained network model ϕ; for the input image I∈R H×W×C Where H represents the image height, W represents the image width, and C represents the number of image channels, features z at different scales are obtained through network ϕ. j =ϕ j (I), where j represents the different layers of the network; multi-scale local features F are obtained by aggregating neighborhood features through adaptive average pooling. j =agg(z j ).

3. The industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction as described in claim 2, characterized in that, The specific expression for step S2 is as follows: Among them, F m Feature maps representing uniform dimensions Indicates a connection operation. Indicates resizing operation, H l W represents the height at its maximum scale. l ϕ represents the width at the maximum scale, and J represents the set of layers used in the network.

4. The industrial image anomaly detection method based on feature-level anomaly synthesis and reconstruction as described in claim 3, characterized in that, In step S3, a feature adapter is used to transform the obtained feature map F m Transferring to the target domain yields a new feature map F. p The specific expression is: Among them, f adapter This is for feature adaptation operations.

5. An industrial image anomaly detection system based on feature-level anomaly synthesis and reconstruction, used to perform the method as described in any one of claims 1-4, characterized in that, Includes the following modules: Image acquisition and feature extraction module: Acquires images and extracts multi-scale local features through a pre-trained network; Unified Feature Map Size Module: Adjusts features of different scales to a unified size, connects features of the unified size, and obtains a feature map; Feature adaptation module: Through feature adaptation, the feature map is transferred to the target domain to obtain a new feature map; Noise addition module: Adds noise to the feature map obtained by the feature adaptation module to obtain a feature map simulating anomalies; Feature reconstruction module: Inputs the feature map obtained by the noise addition module into the feature reconstruction network to reconstruct the features into features without anomalies; Loss calculation module: Calculates the loss by comparing the feature maps before and after adding noise, and trains the feature reconstruction network based on the loss value; The detection result acquisition module uses the trained network to reconstruct features and obtains a segmentation map based on the differences before and after feature reconstruction.

Images