Image defective product detection method and system based on deep learning algorithm

Abstract

The present application relates to the technical field of image defective product detection, in particular to an image defective product detection method and system based on a deep learning algorithm, comprising the following steps: recording sample image information and classifying the recorded data; establishing a deep learning detection model according to the sample image information; obtaining sample image information to be detected and inputting the same into the detection model for identification; the present application can realize an automatic image detection process by establishing a deep learning detection model, without manual intervention, which makes the image detection process more efficient, reduces labor costs, and reduces the influence of human factors on the results; unknown image identification standards are formulated and unknown images are compared; when an unknown image appears, the unknown image is cropped multiple times for identification and detection, and the unknown image is timely classified and processed.

Description

Technical Field

[0001] This invention relates to the field of image defect detection technology, and more specifically, to an image defect detection method and system based on deep learning algorithms. Background Technology

[0002] In product manufacturing, image visual inspection is used to detect product quality. Currently, product inspection only compares the data stored in the recognition model with the product image data. When encountering unknown product image data, it results in the inability to identify the product, causing production line stagnation and affecting production efficiency. At the same time, production managers need to frequently update the data inside the recognition model, increasing management costs and resulting in low efficiency. Therefore, a method and system for detecting defective products based on deep learning algorithms is proposed. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for detecting defective products in images based on deep learning algorithms, so as to solve the problems mentioned in the background art.

[0004] To address the aforementioned technical problems, one objective of this invention is to provide an image defect detection method based on deep learning algorithms, comprising the following steps:

[0005] S1. Record the sample image information and classify the recorded data;

[0006] S2. Establish a deep learning detection model based on sample image information;

[0007] S3. Acquire the image information of the sample to be detected and input it into the detection model for recognition;

[0008] S4. Classify the images that the detection model can directly recognize to generate defective product marking information;

[0009] S5. Receive image information that the detection model fails to recognize and mark it as an unknown image;

[0010] S6. Develop standards for the recognition of unknown images, extract image features from the known parts of unknown images for feature comparison, and classify the unknown images within the detection model based on the comparison results.

[0011] S7. Combine the unknown image and classification processing results and upload them to the sample detection management terminal for learning feedback. Based on the feedback results from the sample detection management terminal, upload the unknown image and classification processing results to the detection model to update the detection model.

[0012] As a further improvement to this technical solution, the step of classifying the recorded data in S1 is as follows:

[0013] S1.1 Establish a data transmission connection channel with the sample testing management terminal;

[0014] S1.2 Extract the image information of the sample and save it in the sample detection management terminal;

[0015] S1.3 Classify the extracted sample image information into normal samples and defective samples.

[0016] As a further improvement to this technical solution, the steps of S2 in establishing the deep learning detection model are as follows:

[0017] Using image information classified as normal samples and defective samples as the training set, a convolutional neural network is selected to extract recognition features from the image information, and then a machine learning algorithm model is used for training to establish a deep learning detection model.

[0018] As a further improvement to this technical solution, the step S3 for acquiring the image information of the sample to be detected is as follows:

[0019] S3.1 Receive the image information of the sample to be tested sent by the detection management terminal through the data transmission connection channel established in S1.1;

[0020] S3.2 Send the image information of the detected sample to the detection model for classification and recognition, and obtain the identifiable image information and the unidentifiable image information.

[0021] As a further improvement to this technical solution, S3.2 sends the identifiable image information to S4, S4 classifies the identifiable image information into normal samples and defective samples according to the identification results, marks the defective samples, and then sends them to the sample detection management terminal to inform it of the identification results. Then S3.2 sends the unidentifiable graphic information to S5.

[0022] As a further improvement to this technical solution, the step of S6, which classifies the unknown image within the detection model based on the comparison results, is as follows:

[0023] S6.1 Select a normal sample and a defective sample for image difference comparison, and formulate the image difference rate obtained from the comparison as the unknown image recognition standard;

[0024] S6.2. Cropping the unknown image and sending the cropped unknown image to the detection model for recognition. If image information matching defective products appears, the unknown image is classified as a defective sample. If no defective sample appears, the image difference between the unknown image and the normal sample image information is compared. If the image difference rate is greater than the position image recognition standard, the unknown image is classified as a defective sample. Conversely, if the image difference rate is less than the position image recognition standard, the unknown image is classified as a normal sample.

[0025] As a further improvement to this technical solution, step S7 sends unknown images classified as normal samples to the sample detection management terminal for identification and determination, with two options: incorrect and correct. Unknown images determined to be defective samples are not sent.

[0026] As a further improvement to this technical solution, the step of S7 in identifying and updating the detection model is as follows:

[0027] S7.1 Record the operational behavior data identified and determined by the sample testing management terminal;

[0028] S7.2. Based on the recorded operation data, upload the unknown image to S2 to update the detection model. If the sample detection management terminal operation is correct, upload the unknown image to the prediction model to update the normal sample image information in the prediction model. If the sample detection management terminal operation is incorrect, upload the unknown image to the prediction model to update the defective sample image information in the prediction model.

[0029] The second objective of this invention is to provide an image defect detection system based on deep learning algorithms, including any one of the above-mentioned image defect detection methods based on deep learning algorithms, comprising a model building unit, an image classification unit, an image comparison unit, and a model updating unit;

[0030] The model building unit is used to record sample image information, classify the recorded data, and build a deep learning detection model based on the sample image information.

[0031] The image classification unit is used to acquire image information of the sample to be detected, input it into the detection model for recognition, classify the images that the detection model can directly recognize, and generate defective product marking information.

[0032] The image comparison unit is used to receive image information that the detection model fails to recognize, mark it as an unknown image, extract the image features of the known parts of the unknown image, formulate the recognition standard for the unknown image, compare the image features of the known parts of the unknown image, and classify the unknown image within the detection model according to the comparison results.

[0033] The model update unit is used to combine the unknown image and the classification processing result and upload them to the sample detection management terminal for learning feedback. Based on the feedback result from the sample detection management terminal, the unknown image and the classification processing result are uploaded to the detection model to identify and update the detection model.

[0034] Compared with existing technologies, the beneficial effects of this invention are as follows: By establishing a deep learning detection model, an automated image detection process can be achieved without manual intervention. This makes the image detection process more efficient, reduces labor costs, and minimizes the impact of human factors on the results. By formulating unknown image recognition standards and comparing unknown images, when an unknown image appears, it is cropped into multiple parts for recognition and detection. Unknown images are promptly classified and processed to avoid situations where unknown product image data cannot be identified, leading to production line stagnation and affecting product production efficiency. At the same time, the image data is saved to the deep learning detection model for data updates, improving the adaptability of this system. Attached Figure Description

[0035] Figure 1 This is an overall flowchart of the present invention;

[0036] Figure 2 This is a flowchart illustrating the process of classifying extracted sample image information into normal samples and defective samples according to the present invention.

[0037] Figure 3 This is a flowchart illustrating the process of acquiring identifiable image information and unidentifiable image information according to the present invention.

[0038] Figure 4 This is a flowchart illustrating the formulation of an unknown image recognition standard for this invention.

[0039] Figure 5 This is a flowchart of the process for updating the image information of defective samples within the prediction model according to the present invention.

[0040] Figure 6 The schematic diagram of the model building unit of this invention is shown.

[0041] The meanings of the labels in the diagram are as follows:

[0042] 10. Model building unit; 20. Image classification unit; 30. Image comparison unit; 40. Model update unit. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example 1

[0045] like Figures 1-6As shown, one of the objectives of this invention is to provide an image defect detection method based on a deep learning algorithm, comprising the following steps:

[0046] S1. Record the sample image information and classify the recorded data;

[0047] The steps S1 takes to classify the recorded data are as follows:

[0048] S1.1 Establish a data transmission connection channel with the sample testing management terminal; the steps are as follows:

[0049] Determine the communication protocol: Use HTTP as the communication protocol between this system and the product testing management terminal;

[0050] Design a data transmission interface: Based on the data interaction requirements between the sample testing management terminal and this system, design a corresponding data transmission interface. The interface may include data acquisition, data transmission, and data feedback functions.

[0051] S1.2 Extract the image information of the sample and save it in the sample detection management terminal;

[0052] Determine the image storage location: First, it is necessary to determine the storage location where the sample images are saved by the sample detection management terminal;

[0053] Access storage location: Use the file operation API to access image information and copy and save it.

[0054] S1.3. Classify the extracted sample image information into normal samples and defective samples. Use semi-supervised learning to label or classify each extracted sample image information, that is, mark it as a normal sample or a defective sample.

[0055] S2. Establish a deep learning detection model based on sample image information;

[0056] The steps for building a deep learning detection model using S2 are as follows:

[0057] Using image information categorized as normal samples and defective samples as the training set, a convolutional neural network is selected to extract recognition features from the image information. Then, a machine learning algorithm model is used for training to establish a deep learning detection model. The expression is:

[0058] Input image representation: Represent the input image as a tensor. Where the dimension of x is Represents the height of the image Indicates the width of the image. Indicates the number of channels in an image;

[0059] Convolutional feature extraction: This involves extracting features from the input image using convolutional layers. For deep learning detection models, multiple convolutional and pooling layers are typically stacked to extract both low-level and high-level features of the image.

[0060] Convolutional layers: These layers use convolution kernels to perform convolution operations on the input image, resulting in convolutional feature maps. Each convolutional layer typically consists of multiple convolution kernels, each extracting a specific type of feature.

[0061] Activation function: Apply an activation function, such as the ReLU function, after the convolutional layer to set the negative values ​​in the convolutional feature map to zero, introducing non-linear features;

[0062] Pooling layer: Downsamples the convolutional feature map, reduces the number of parameters, and extracts more robust features;

[0063] Fully connected layer: Flattens the convolutional feature map into a one-dimensional vector and maps it to a low-dimensional space. A fully connected layer typically consists of multiple neurons, each connected to all neurons in the previous layer.

[0064] Output layer: A suitable sigmoid function is used to map the output of the fully connected layer to a probability distribution. Depending on the task requirements, the output layer typically includes two neurons, representing the probability of a normal sample and the probability of a defective sample, respectively.

[0065] Model training: Using labeled image data as the training set, the model is trained and the model parameters are optimized through backpropagation algorithm and loss function.

[0066] S3. Acquire the image information of the sample to be detected and input it into the detection model for recognition;

[0067] The steps for S3 to acquire image information of the sample to be tested are as follows:

[0068] S3.1 Receive the image information of the sample to be tested sent by the sample testing management terminal through the data transmission connection channel established in S1.1; the steps are as follows:

[0069] Waiting to receive data: On the receiving end, create a listener program or thread to wait for data to be received. This program will continuously listen on the specified port, waiting for the sample detection management end to send image information;

[0070] Receiving image data: When image data is sent to the receiving end, the image data is received through the data transmission connection channel.

[0071] S3.2. Send the sample image information to the detection model for classification and recognition, obtaining identifiable and unidentifiable image information. The formula is:

[0072] Assume there is The probability distribution of each category obtained after the input image x is processed by the deep learning detection model is as follows:

[0073]

[0074] in, This represents the probability that the input image x belongs to category 1. Let x represent the probability that the input image x belongs to category 2. Then the classification result is: if the probability distribution shows that a certain category has the highest probability, then... If it is the largest, then the image is identified as that category;

[0075] If the probabilities of all categories in the probability distribution are below the set threshold, or if an error occurs during the classification process, the image is deemed unrecognizable.

[0076] S3.2 sends the recognizable image information to S4, and S3.2 sends the unrecognizable image information to S5.

[0077] S4. Classify the images that the detection model can directly recognize and generate defective product marking information; S4 classifies the recognizable image information into normal samples and defective samples according to the recognition results, marks the defective samples, and then sends them to the sample detection management terminal to inform the recipient of the recognition results.

[0078] S5. Receive image information that the detection model failed to recognize and mark it as an unknown image; receive image information sent by S3.2 and mark it.

[0079] S6. Develop standards for the recognition of unknown images, extract image features from the known parts of unknown images for feature comparison, and classify the unknown images within the detection model based on the comparison results.

[0080] S6 The steps for classifying unknown images within the detection model based on the comparison results are as follows:

[0081] S6.1. Select a normal sample and a defective sample for image difference comparison, and use the image difference rate obtained from the comparison as the standard for unknown image recognition; the steps are as follows:

[0082] Sample selection: Select one sample from the normal sample and one sample from the defective sample as a reference sample;

[0083] Image difference calculation: Compare the unknown image with a reference sample and calculate the difference between them. The formula is as follows:

[0084] Mean Squared Error (MSE):

[0085] ;

[0086] in, and These represent the height and width of the image, respectively. and Representing the coordinates of the reference sample and the unknown image respectively. Pixel value at;

[0087] Normalization: Normalize the image differences, mapping them to the range [0, 1], and converting them into a difference rate;

[0088] Establish standards for unknown image recognition: Based on the difference rate, establish standards for the recognition of unknown images.

[0089] S6.2. Cropping the unknown image and sending it to the detection model for identification. If image information matching defective products appears, the unknown image is classified as a defective sample. If no defective sample appears, the image difference between the unknown image and the normal sample image is compared. If the image difference rate is greater than the position image recognition standard, the unknown image is classified as a defective sample. Conversely, if the image difference rate is less than the position image recognition standard, the unknown image is classified as a normal sample. The steps are as follows:

[0090] Image cropping: Cropping unknown images to obtain regions that need to be classified and identified;

[0091] Send to the detection model: The cropped unknown image is sent to the detection model for identification. The model calculates the probability distribution for each category;

[0092] Defective sample identification: If the category corresponding to the highest probability of the detection model is a defective sample, then this unknown image is classified as a defective sample.

[0093] Image difference comparison: If the detection model does not identify the sample as defective, the unknown image is compared with the normal sample image information by image difference comparison;

[0094] Classification: The image is compared with a set threshold image recognition standard, and the classification of the unknown image is determined based on the comparison result.

[0095] If the difference rate is greater than the threshold, this unknown image is classified as a defective sample.

[0096] If the difference rate is less than or equal to the threshold, the unknown image is classified as a normal sample.

[0097] S7. Combine the unknown image and classification processing results and upload them to the sample detection management terminal for learning feedback. Based on the feedback results from the sample detection management terminal, upload the unknown image and classification processing results to the detection model to update the detection model.

[0098] S7 sends unknown images classified as normal samples to the sample testing management terminal for identification and determination, offering two options: incorrect and correct. Unknown images of samples determined to be defective are not sent. A questionnaire is sent to the sample testing management terminal, with the questionnaire selecting "incorrect" or "correct."

[0099] The steps for S7 to update the detection model are as follows:

[0100] S7.1 Record the operational behavior data identified and determined by the sample testing management terminal; obtain the selected options in the questionnaire content. If the selection is correct, the operational behavior data is correct; if the selection is incorrect, the operational behavior data is incorrect.

[0101] S7.2. Based on the recorded operation data, upload the unknown image to S2 for detection model update. If the sample detection management terminal operation is correct, upload the unknown image to the prediction model to update the normal sample image information in the prediction model. If the sample detection management terminal operation is incorrect, upload the unknown image to the prediction model to update the defective sample image information in the prediction model. The steps are as follows:

[0102] Determine the target to be updated: Based on the correctness and error of the sample testing management terminal operation, determine whether the target to be updated is the normal sample image information or the defective sample image information in the prediction model.

[0103] Data preprocessing: Perform necessary data preprocessing on the uploaded unknown images to make them meet the input requirements of the prediction model, such as image resizing and normalization;

[0104] Update the dataset: Add the preprocessed unknown image to the corresponding dataset. If the sample detection management terminal operation is correct, the unknown image is added to the dataset of normal sample image information; if the operation is incorrect, the unknown image is added to the dataset of defective sample image information.

[0105] Model training: Use the updated dataset to train the model using a deep learning model;

[0106] Model evaluation: Evaluate the performance and accuracy of the retrained prediction model using a validation set or other evaluation methods to ensure the quality of the updated model;

[0107] Model update: Replace the original prediction model with a newly trained model to update the model's capabilities and accuracy.

[0108] The second objective of this invention is to provide an image defect detection system based on a deep learning algorithm, including any one of the above-mentioned image defect detection methods based on a deep learning algorithm, comprising a model building unit 10, an image classification unit 20, an image comparison unit 30, and a model updating unit 40.

[0109] The model building unit 10 is used to record sample image information, classify the recorded data, and build a deep learning detection model based on the sample image information;

[0110] The image classification unit 20 is used to acquire image information of the sample to be detected, input it into the detection model for recognition, classify the images that the detection model can directly recognize, and generate defective product marking information.

[0111] The image comparison unit 30 is used to receive image information that the detection model fails to recognize, mark it as an unknown image, extract the image features of the known parts of the unknown image, formulate the recognition standard for the unknown image, compare the image features of the known parts of the unknown image, and classify the unknown image within the detection model according to the comparison results.

[0112] The model update unit 40 is used to combine the unknown image and the classification processing result and upload them to the sample detection management terminal for learning feedback. Based on the feedback result from the sample detection management terminal, the unknown image and the classification processing result are uploaded to the detection model to identify and update the detection model.

[0113] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting defective products in images based on deep learning algorithms, characterized in that: Includes the following steps: S1. Record the sample image information and classify the recorded data; S2. Establish a deep learning detection model based on sample image information; S3. Acquire the image information of the sample to be detected and input it into the detection model for recognition; S4. Classify the images that the detection model can directly recognize to generate defective product marking information; S5. Receive image information that the detection model fails to recognize and mark it as an unknown image; S6. Develop standards for the recognition of unknown images, extract image features from the known parts of unknown images for feature comparison, and classify the unknown images within the detection model based on the comparison results. S7. Combine the unknown image and classification processing results and upload them to the sample detection management terminal for learning feedback. Based on the feedback results from the sample detection management terminal, upload the unknown image and classification processing results to the detection model to identify and update the detection model.

2. The image defect detection method based on deep learning algorithm according to claim 1, characterized in that: The steps of S1 in classifying the recorded data are as follows: S1.1 Establish a data transmission connection channel with the sample testing management terminal; S1.2 Extract the image information of the sample and save it in the sample detection management terminal; S1.3 Classify the extracted sample image information into normal samples and defective samples.

3. The image defect detection method based on deep learning algorithm according to claim 1, characterized in that: The steps for establishing the deep learning detection model in S2 are as follows: Using image information classified as normal samples and defective samples as the training set, a convolutional neural network is selected to extract recognition features from the image information, and then a machine learning algorithm model is used for training to establish a deep learning detection model.

4. The image defect detection method based on deep learning algorithm according to claim 2, characterized in that: The steps for S3 to acquire the image information of the sample to be detected are as follows: S3.1 Receive the image information of the sample to be tested sent by the detection management terminal through the data transmission connection channel established in S1.1; S3.2 Send the image information of the detected sample to the detection model for classification and recognition, and obtain the identifiable image information and the unidentifiable image information.

5. The image defect detection method based on deep learning algorithm according to claim 4, characterized in that: S3.2 sends the identifiable image information to S4. S4 classifies the identifiable image information into normal samples and defective samples according to the identification results, marks the defective samples, and then sends them to the sample testing management terminal to inform it of the identification results. Then S3.2 sends the unidentifiable graphic information to S5.

6. The image defect detection method based on deep learning algorithm according to claim 1, characterized in that: The step S6, which involves classifying the unknown image within the detection model based on the comparison results, is as follows: S6.1 Select a normal sample and a defective sample for image difference comparison, and formulate the image difference rate obtained from the comparison as the unknown image recognition standard; S6.

2. Cropping the unknown image and sending the cropped unknown image to the detection model for recognition. If image information matching defective products appears, the unknown image is classified as a defective sample. If no defective sample appears, the image difference between the unknown image and the normal sample image information is compared. If the image difference rate is greater than the position image recognition standard, the unknown image is classified as a defective sample. Conversely, if the image difference rate is less than the position image recognition standard, the unknown image is classified as a normal sample.

7. The image defect detection method based on deep learning algorithm according to claim 1, characterized in that: S7 sends unknown images classified as normal samples to the sample detection management terminal for identification and determination, with two options: incorrect and correct. Unknown images determined to be defective samples are not sent.

8. The image defect detection method based on deep learning algorithm according to claim 1, characterized in that: The steps for S7 to identify and update the detection model are as follows: S7.1 Record the operational behavior data identified and determined by the sample testing management terminal; S7.

2. Based on the recorded operation data, upload the unknown image to S2 to update the detection model. If the operation of the sample detection management terminal is correct, upload the unknown image to the prediction model to update the normal sample image information in the prediction model. If the operation of the sample detection management terminal is incorrect, upload the unknown image to the prediction model to update the defective sample image information in the prediction model.

9. A system for detecting defective products in images based on deep learning algorithms, comprising the image defective product detection method based on deep learning algorithms as described in any one of claims 1-8, characterized in that: It includes a model building unit (10), an image classification unit (20), an image comparison unit (30), and a model update unit (40); The model building unit (10) is used to record sample image information, classify the recorded data, and build a deep learning detection model based on the sample image information; The image classification unit (20) is used to acquire the image information of the sample to be detected, input it into the detection model for recognition, classify the images that the detection model can directly recognize, and generate defective product marking information. The image comparison unit (30) is used to receive image information that the detection model fails to recognize, mark it as an unknown image, extract the image features of the known part of the unknown image, formulate the recognition standard of the unknown image, compare the image features of the known part of the unknown image, and classify the unknown image in the detection model according to the comparison results. The model update unit (40) is used to combine the unknown image and the classification processing result and upload them to the sample detection management terminal for learning feedback, and upload the unknown image and the classification processing result to the detection model according to the feedback result of the sample detection management terminal, and update the detection model for recognition.

Images