Industrial defect detection method based on memory bank
By adopting memory bank-based neural network technology in industrial defect detection, the problem of traditional methods taking time and low accuracy is solved, and efficient and accurate defect detection is achieved.
Patent Information
- Application Number
- CN202411687842.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-18
AI Technical Summary
Traditional industrial defect detection methods are time-consuming, have low accuracy and high false positive rate, which cannot effectively improve detection efficiency and accuracy.
Using the industrial defect detection technology based on memory database, a pre-trained neural network is used to extract the training images, and the features and position information are aggregated and stored in the memory database, and the features are updated as the production process is performed. The feature extraction of the image to be detected is compared with the features in the memory bank to identify defects.
Automatic defect detection is realized, reducing the time and errors of manual detection, improving the accuracy and efficiency of detection, and reducing the false positive rate.
Smart Images
Figure CN119648645A_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the field of machine learning technology, and in particular to an industrial defect detection technology, electronic equipment and storage medium based on a memory library. [Background Technology]
[0002] Before the emergence of artificial intelligence and neural networks, traditional defect detection methods mostly used methods that directly compared images with template images after image processing, assisted by manual re-inspection, which had problems such as being time-consuming, low in accuracy, and a high false positive rate. [Summary of the invention]
[0003] The problems existing in the prior art solved by this application are:
[0004] Traditional defect detection methods mostly use the method of directly comparing the image with the template image after image processing, assisted by manual re-inspection, which has problems such as being time-consuming, low in accuracy, and high false positive rate.
[0005] To achieve the above objectives, in a first aspect, the present application provides an industrial defect detection technology based on a memory library, and the industrial defect detection technology based on a memory library includes:
[0006] Using a pre-trained neural network to perform feature extraction on the training image to obtain features of the image;
[0007] Aggregate features and location information and store them in a memory bank, and update the stored features as the production process progresses;
[0008] The image to be inspected and the annotation box are obtained, and features of the annotation box and the image to be inspected are extracted and compared with the features in the memory library to identify defects in the image to be inspected.
[0009] In some embodiments, the step of extracting features from the image to be trained using a pre-trained neural network to obtain features of the image includes:
[0010] Select different layers of different pre-trained models and generate corresponding feature extractors for feature extraction, such as:
[0011] Layer1, Layer2, Layer3 of WideResnet50;
[0012] Layer1, Layer2, Layer3 of CSPResnet50;
[0013] EfficientNet’s Blocks.1, Blocks.2, etc.
[0014] In some embodiments, the aggregating the features and the location information and storing them, and updating the stored features as the production process proceeds, includes:
[0015] The features of different layers extracted from the image by the feature extractor are enlarged to the same size for aggregation, and are projected to a specified dimension through average pooling for storage as a memory bank.
[0016] In some embodiments, the method further comprises:
[0017] After the features and location information are aggregated and stored, the stored features are updated in different ways, including weighted averaging or partial replacement of old and new features.
[0018] In some embodiments, the method further comprises:
[0019] By analyzing the distribution of features in the memory library, a threshold for the anomaly score is calculated.
[0020] In some embodiments, the step of acquiring the image to be detected and the marked frame, extracting features from the marked frame and the image to be detected therein, and comparing the features with the features in the memory library to identify defects in the image to be detected includes:
[0021] The features extracted by the feature extractor from the marked box and the image to be detected therein are aggregated and compared with the features in the memory library, and an anomaly score is calculated using its most recent features and compared with a threshold of the score to determine whether there are defects in the image to be detected in the marked box.
[0022] In some embodiments, after identifying the abnormal annotation frame, the defect detection method further includes:
[0023] Generate defect heat map based on feature comparison to visualize defect location;
[0024] Feedback the anomaly score according to the defect annotation box;
[0025] In a second aspect, the present application provides an electronic device comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors are used to execute the one or more computer programs stored in the memory, so that the one or more processors perform an automatic quality inspection method for labeling information as described in any one of the first aspects.
[0026] In a third aspect, the present application provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the method as described in any one of the first aspects.
[0027] Implementing one of the above technical solutions of this application has the following advantages or beneficial effects:
[0028] The memory-based industrial defect detection technology, electronic device and storage medium of the present application can automatically train the training images and perform defect detection on the subsequent images to be detected under the operation of the operator. First, the pre-trained neural network is used to extract features of the training image to obtain the features of the image, and then the features and position information are aggregated and stored in the memory, and the stored features are updated as the production process proceeds; finally, the image to be detected and the annotation box are obtained, and the features of the annotation box and the image to be detected in it are extracted, and compared with the features in the memory to identify defects in the image to be detected.
[0029] This method can detect defects without the need for manual inspection one by one, and has high accuracy, thereby improving the efficiency and effect of defect detection. [Drawings]
[0030] Figure 1 It is a flow chart of an industrial defect detection method based on a memory library of the present invention;
[0031] Figure 2 It is a schematic framework diagram of an electronic device of the present invention. [Specific implementation method]
[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.
[0033] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0034] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.
[0035] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there can be a central component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there can be a central component at the same time.
[0036] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0037] The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0038] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0039] See also Figure 1 In the first embodiment of the present invention, the memory-based industrial defect detection method includes the following steps S101 to S103.
[0040] S101: extracting features from a training image using a pre-trained neural network to obtain features of the image;
[0041] In the field of machine learning, a pre-trained neural network refers to a neural network that has been trained on a large dataset and has learned certain common features. Afterwards, these networks can be used to handle similar tasks, saving computing resources and time. The output of each layer of these pre-trained neural networks contains feature information of different scales of the image.
[0042] In some of the embodiments, the pre-trained neural network and the layers used may be pre-trained on ImageNet:
[0043] Layer1, Layer2, Layer3 of WideResNet50;
[0044] Layer1, Layer2, Layer3 of CSPResNet50;
[0045] Blocks.1, Blocks.2, etc. of EfficientNetB4.
[0046] Among them, ResNet is a deep convolutional neural network proposed by Kaiming He and others from Microsoft Research in 2015, and won the championship in the image classification task in the ImageNet 2015 competition. The core innovation of ResNet is the introduction of residual connections, which effectively solves the gradient vanishing and degradation problems in deep networks, making it easier to deepen the network and improve the performance of the model.
[0047] CSP ResNet is an improved neural network architecture that combines the ideas of ResNet (residual network) and CSPNet (Cross Stage Partial Network). It is mainly used for computer vision tasks such as object detection and image classification. Its goal is to improve the computational efficiency and performance of the network while reducing the computational complexity of the model.
[0048] Effici entNet is a convolutional neural network architecture proposed by Google in 2019, which aims to achieve higher accuracy and lower computational cost through efficient network design. Compared with traditional deep learning models (such as ResNet and VGG), Effici entNet has significantly reduced parameters and computational complexity, but has better performance.
[0049] S102: Aggregate the features and position information and store them in a memory bank, and update the stored features as the production process proceeds;
[0050] After obtaining the features of the training image, the location information is added and stored in the memory bank.
[0051] Memory Bank is a mechanism for storing and managing model features, which is widely used in self-supervised learning, contrastive learning, and reinforcement learning. It stores the features generated by the model in external storage so that they can be efficiently compared and reused during training, reducing computing costs. Memory Bank can handle large-scale data, improve training efficiency, and support the model to maintain feature consistency between different batches.
[0052] In some of the embodiments, multiple layers of a neural network are used to extract multi-scale features. In this case, the information of each layer needs to be enlarged to the same size and aggregated using methods including but not limited to average pooling and splicing, and its feature channels are projected to a certain number of feature channels through linear transformation.
[0053] In some implementations, as the production process progresses, the features in the original memory bank can no longer accurately represent the features of normal samples. At this time, it is necessary to update the features in the memory bank in some way, including but not limited to partial replacement, weighted average, directly adding new samples to the memory bank, etc.
[0054] In some embodiments, the method further comprises:
[0055] By analyzing the distribution of features in the memory library, a threshold for the anomaly score is calculated.
[0056] Specifically, the features in the memory bank have a certain distribution, and the obtained score also conforms to a certain distribution. Approximately, if the values in each feature channel in the tensor are considered to conform to the normal distribution, the approximate abnormal score threshold can be inferred by the three-sigma principle of the normal distribution.
[0057] S103: Obtain the image to be detected and the annotation box, extract features from the annotation box and the image to be detected therein, and compare them with the features in the memory library to identify defects in the image to be detected.
[0058] In machine learning tasks, a label box refers to a rectangular area used to locate an object in an image. The label box is usually used in object detection tasks to indicate the position and size of the object identified by the model in the image.
[0059] The annotation box is usually represented by four coordinates, namely the x and y coordinates of the upper left corner, and the x and y coordinates of the lower right corner. These four coordinates define the position and size of the annotation box, allowing the machine learning model to identify and locate specific objects in the image.
[0060] In the present application, only a portion of the image to be detected may need to be detected, so in some implementations, feature extraction is performed only on the portion marked by the annotation box to save time for feature extraction.
[0061] The features extracted from the training image and the image to be tested are tensors with the same scale and the same number of feature channels. Comparing the two means quantifying the difference between the two tensors through different functions. These functions include but are not limited to: Euclidean distance, Mahalanobis distance, cosine similarity, and Manhattan distance.
[0062] In some embodiments, after identifying the abnormal annotation frame, the defect detection method further includes:
[0063] Generate defect heat map based on feature comparison to visualize defect location;
[0064] Feedback the anomaly score according to the defect annotation box.
[0065] Specifically, each position will be given a score after feature comparison to characterize the abnormal value of the position. The score is compared with the threshold value obtained in S102 to obtain the abnormality of the image to be tested. The abnormal scores of all positions are summarized to obtain a defect heat map.
[0066] Among them, the heat map is a data visualization tool used to show the distribution and intensity of data in two-dimensional space. It uses color gradients or concentrations to represent the magnitude of values at different positions, and is usually used to intuitively reveal the changing trends and patterns of values in space. The operator can intuitively see the location and intensity of defects in the image to be inspected through the heat map.
[0067] See also Figure 2 In the second embodiment of the present invention, the present application further provides an electronic device 1, comprising: one or more processors 11; a memory 10, for storing one or more computer programs, wherein the one or more processors 11 are used to execute the one or more computer programs stored in the memory 10, so that the one or more processors 11 execute an automatic quality inspection method for labeling information as described in any one of the first embodiments.
[0068] The electronic device 1 may be a series of electronic devices such as a smart phone, a tablet computer, a wearable electronic device, a smart home electronic product, and an industrial device.
[0069] Since the implementation of the electronic device 1 is described in detail in the first embodiment, it will not be repeated here.
[0070] The electronic device 1 of the embodiment of the present application significantly reduces the time required for manual quality inspection through automated processing, speeds up the data processing process, and improves the efficiency of defect detection; and, by using advanced neural network technology, it can more accurately identify defects, reduce human omissions, and enhance the accuracy of quality inspection; by using the method described in the present application to perform defect detection, there is no need for manual inspection one by one, and it has high accuracy, improving the efficiency and effect of defect detection. Moreover, the present application can also reduce the reliance on manual review, saving a lot of manpower and time costs.
[0071] The present application also provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the method as described in any one of the first embodiments.
[0072] Those skilled in the art can understand that all or part of the features / steps of the above-mentioned method embodiments can be implemented by methods, data processing systems or computer programs, and these features can be implemented without hardware, entirely by software, or by a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media, and the storage medium stores a computer program, and when the computer program is executed (such as a processor), the steps of the automatic quality inspection method embodiment including the above-mentioned annotation information are executed.
[0073] The aforementioned storage media that can store program codes include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks and / or combinations of the above devices, that is, it can be implemented by any type of volatile or non-volatile storage device or a combination thereof.
[0074] Compared with the prior art, the present invention provides an industrial defect detection method 1 based on a memory library. The method performs defect detection without the need for manual detection one by one, and has high accuracy, thereby improving the efficiency and effect of defect detection.
[0075] The above-described embodiments of the present invention do not constitute a limitation on the protection scope of the present invention. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.
Claims
1. An industrial defect detection method based on a memory library, characterized in that: The following steps are included: S1: extracting features of the image to be trained using a pre-trained neural network to obtain features of the image to be trained; S2: Aggregating the features and position information of the image to be trained and storing them in a memory bank, and updating the stored features as the production process proceeds; S3: Acquire the image to be detected and the marked frame, extract features from the marked frame and the image to be detected therein, and compare them with the features stored in the memory bank to identify defects in the image to be detected.
2. The memory-based industrial defect detection method according to claim 1, characterized in that: In step S1, the pre-trained neural network is used to extract features from the image to be trained, and the features of the image to be trained are obtained, which specifically includes: selecting different layers of different pre-trained models and generating corresponding feature extractors for feature extraction, and the different layers include but are not limited to: Layer1, Layer2, Layer3 of WideResnet50; Layer1, Layer2, Layer3 of CSPResnet50; Blocks.1 and Blocks.2 of EfficientNet.
3. The memory-based industrial defect detection method according to claim 2, characterized in that: In the step S2, the features and position information of the image to be trained are aggregated and stored, and the stored features are updated as the production process proceeds, including the following step S4: the features of different layers extracted from the image by the feature extractor are enlarged to the same size for aggregation, and are projected to a specified dimension through average pooling for storage as a memory library.
4. The memory-based industrial defect detection method according to claim 1, characterized in that: Also includes: In the step S2, after the features and the position information are aggregated and stored, the stored features are updated in different ways, including the following step S5: updating the new and old features by weighted averaging or partial replacement.
5. The memory-based industrial defect detection method according to claim 3, characterized in that: The method also includes step S6: calculating a threshold of anomaly score by analyzing the distribution of features in the memory library.
6. The memory-based industrial defect detection method according to claim 3, characterized in that: In the step S3, the image to be detected and the marked frame are obtained, and features of the marked frame and the image to be detected therein are extracted, and compared with the features in the memory library to identify defects in the image to be detected, specifically including aggregating the features extracted by the feature extractor from the marked frame and the image to be detected therein according to step S4, and comparing them with the features in the memory library, and calculating the anomaly score through its most recent features, and comparing it with the threshold of the score in step S6 to determine whether there are defects in the image to be detected in the marked frame.
7. The memory-based industrial defect detection method according to claim 1, characterized in that: After the abnormality annotation frame is identified, the defect detection method further includes generating a defect heat map based on feature comparison, visually displaying the defect location, and feeding back an abnormality score according to the defect annotation frame.
8. An electronic device, characterized in that: comprising one or more processors; A memory for storing one or more computer programs, and one or more processors for executing one or more computer programs stored in the memory, so that one or more processors execute an automatic quality inspection method for labeling information as described in any one of claims 1-7.
9. A non-volatile computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 7.