Production process defect detection and identification system based on deep learning

By combining convolutional neural networks and recursive neural networks, combined with self-attention mechanisms and data enhancement technology, the shortcomings of existing defect detection systems in positioning accuracy and complex image processing are solved, and accurate detection and positioning of defects are achieved, improving the improvement of the problem. The comprehensiveness and accuracy of the detection.

CN119068252AInactive Publication Date: 2024-12-03BEIJING RICHAUTO S&T CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411154938.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing defect detection system has shortcomings in positioning accuracy and complex image processing, and cannot effectively combine the spatial and timing characteristics of the image, resulting in insufficient detection accuracy and efficiency.

Method used

Deep learning-based system is adopted, combining convolutional neural networks and recurrent neural networks for defect detection, and using a self-attention mechanism in the defect positioning module, high-quality synthetic defect images are generated through the data augmentation module, and the adaptive learning module is based on production Environmental adjustment parameters.

Benefits of technology

Accurate detection and positioning of defects is achieved, comprehensiveness and accuracy of detection is improved, misjudgment and missed inspection are reduced, and production efficiency and product quality are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068252A_ABST
    Figure CN119068252A_ABST
Patent Text Reader

Abstract

The invention discloses a production process defect detection and identification system based on deep learning, and the system comprises an image data obtaining module which is used for obtaining the image data of a product; and the data enhancement module is used for receiving the image data generated by the image data acquisition module and preprocessing the image data to generate a high-quality synthetic defect image as a training set of the training model. According to the invention, the defect detection module combines the advantages of the convolutional neural network and the recurrent neural network, can efficiently extract the spatial features and time sequence features of the image, achieves accurate defect detection and classification, and further accurately positions the defect area by using the self-attention mechanism in the defect positioning module. Specifically, the convolutional neural network is good at processing the spatial features of the image, extracts the edge, texture and complex structure information of the image through a plurality of convolutional layers, and captures the low-level and high-level features of the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of defect detection and identification, and specifically, relates to a production process defect detection and identification system based on deep learning. Background Art

[0002] In modern industrial production, product quality control and assurance are of vital importance. Defect detection and identification systems in the production process are key technologies for achieving efficient quality management. Defect detection systems can detect and identify various defects in products in a timely manner by real-time monitoring and analysis of products produced during the production process, thereby ensuring high product quality and consistency. This not only helps to reduce scrap rates and improve production efficiency, but also reduces production costs and enhances the market competitiveness of enterprises.

[0003] Visual inspection is the most common defect detection method. Using high-resolution cameras and image processing technology, the product surface is scanned and analyzed. Through image processing algorithms such as edge detection, texture analysis and shape recognition, scratches, cracks, dents and other defects on the product surface are identified. Non-destructive testing technologies include ultrasonic testing, X-ray testing and magnetic particle testing. These methods identify internal defects such as pores, cracks and internal unevenness by analyzing the internal structure of the product. These technologies can perform inspections without destroying the product to ensure the integrity of the product.

[0004] Existing technologies have obvious limitations in defect location. They lack precise location methods and can only roughly mark the defect location. This rough location method results in the inability to accurately determine the specific location and scope of the defect in actual applications, which in turn affects the subsequent defect repair and processing efficiency. Due to the lack of detailed location information, maintenance personnel may need more time and resources to confirm and handle defects, which not only increases the complexity of operations, but may also lead to omissions or misjudgments.

[0005] At the same time, in the process of defect detection, the existing technology usually only uses a single type of neural network for processing, which is unable to take into account both the spatial features and the temporal features of the image at the same time. The limitation of this single network structure leads to its poor performance when processing complex image data. Although the convolutional neural network (CNN) has a strong ability in extracting image spatial features, it lacks analysis of temporal features. While the recurrent neural network (RNN) has advantages in processing temporal data, it is less efficient when processing high-dimensional spatial features. Due to the inability to effectively combine the advantages of these two networks, the existing detection systems often lack detection accuracy and efficiency when faced with complex defect scenarios. This not only affects the comprehensiveness and accuracy of defect detection, but may also lead to missed detections or false detections, which in turn affects the overall quality and production efficiency of the product.

[0006] In view of this, the present invention is proposed. Summary of the invention

[0007] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a production process defect detection and identification system based on deep learning, which solves the problems raised in the above-mentioned background technology.

[0008] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:

[0009] A production process defect detection and identification system based on deep learning, comprising: an image data acquisition module, the image data acquisition module is used to acquire image data of a product;

[0010] A data enhancement module, which is used to receive the image data generated by the image data acquisition module and pre-process the image data to generate a high-quality synthetic defect image as a training set for the training model;

[0011] Defect detection module, which uses trained convolutional neural networks and recurrent neural networks to extract image features and analyze time series features;

[0012] Defect localization module: The defect localization module is used to receive the output results of the defect analysis module and use the self-attention mechanism to locate the defect position of the product;

[0013] Adaptive learning module: The adaptive learning module can automatically adjust parameters according to changes in the production environment, and continuously learn and optimize.

[0014] Optionally, when acquiring image data of a product, the image data acquisition module uses a camera to capture an image of the product and transmits the captured image data to the processing unit.

[0015] Optionally, the data enhancement module is used to receive the image data generated by the image data acquisition module and pre-process the image data to generate a high-quality synthetic defect image in the following steps:

[0016] Receive the preprocessed image data from the image data acquisition module and adjust the brightness, contrast, saturation and hue of the image by the formula I′=α·I+β, where I is the original image, I′ is the adjusted image, and α and β are brightness and contrast adjustment parameters;

[0017] Add random noise and defects to the image, including scratches, stains, and cracks;

[0018] By formula Generate images with multiple defects superimposed, simulate complex scenes, and provide high-quality data for training models. final is the final generated multi-defect superposition image, α iis the transparency of the ith defect, is the i-th defect image, I base is the image after adjusting brightness and contrast, n is the number of defect types;

[0019] Sum the images generated by all processing steps.

[0020] Optionally, the defect detection module uses the trained convolutional neural network and recurrent neural network to extract image features and analyze time series features in the following steps:

[0021] Obtain preprocessed image data from the image data acquisition module, and use a convolutional neural network to extract spatial features of the image;

[0022] Using recurrent neural networks to The features extracted are used for time series analysis, and the outputs of convolutional neural networks and recurrent neural networks are combined for defect recognition and classification.

[0023] Map the timing features to different defect categories and calculate the confidence score of each defect category through the formula to determine the defect type

[0024] The identified defect categories and corresponding features are reversely mapped to the original image to mark the defective areas.

[0025] Optionally, the convolutional neural network includes an input layer, a multi-layer convolutional layer, an activation layer, a pooling layer, and a fully connected layer. The steps of extracting the spatial features of an image using the convolutional neural network are:

[0026] The image I is input into the convolutional neural network model and multiple convolutional layers are used to extract the low-level and high-level features of the image, which are expressed as: Among them, F l is the feature map of the lth layer, φ is the activation function ReLU, K l is the number of convolution kernels in layer l, W lk is the weight of the kth convolution kernel in the lth layer, F l-1 is the input feature map of the l-1th layer, b lk is the bias of the kth convolution kernel in the lth layer, and * represents the convolution operation;

[0027] The ReLU activation function is used to increase the nonlinearity of the network, and the pooling layer is used to reduce the dimension of the feature map, reduce the amount of calculation, and retain important features. The extracted features are further processed through the fully connected layer to output a fixed-length feature vector, which is expressed as: f is the output fixed-length feature vector, n is the dimension of the flattened feature map, and w i is the weight of the fully connected layer, p i is the flattened output feature of the pooling layer, and b is the bias of the fully connected layer.

[0028] Optionally, the steps of performing time series analysis on the extracted features using a recurrent neural network are:

[0029] The feature vectors extracted by the convolutional neural network are input into the recurrent neural network in the form of time series. The recurrent neural network is used to process the input sequence to capture the temporal dependency between the feature vectors. The expression is: t =φ(W h h t-1 +W x f t +b h ), where ht is the hidden state at the tth time step, W h is the weight matrix of the hidden state, W x is the weight matrix of the input feature vector, b h is the bias of the hidden state; the recurrent neural network saves and updates the timing information through its internal state. Finally, the output layer of the recurrent neural network generates the final timing feature representation.

[0030] Optionally, the defect location module is used to receive the output results of the defect analysis module and use the self-attention mechanism to locate the defect position of the product:

[0031] Receive the defect detection results output by the defect analysis module and extract feature maps from the original image or intermediate layer, and configure the number of attention heads and embedding dimension of the self-attention model;

[0032] By formula Calculate the attention weight of each pixel in the feature map and generate a weighted attention feature map based on the attention weight. Identify the defect location based on the high-weight area in the attention feature map, where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector;

[0033] Use boundary detection algorithm to determine the specific boundary of defect area and mark the defect area on the original image to show the specific location and size of the defect;

[0034] Summarize defect category, location, size and confidence information to form a complete positioning result.

[0035] Optionally, the adaptive learning module can automatically adjust parameters according to changes in the production environment. The steps for continuous learning and optimization are:

[0036] Collect data and defect detection results from the production environment, and detect changing trends and anomalies in the environmental data;

[0037] Based on the extracted features and historical data, a parameter adjustment strategy is generated using random forest to determine how the parameters of the detection model should be adjusted;

[0038] Based on the latest environmental data and defect detection results, update the training dataset and use the validation dataset and actual production data to evaluate the model's detection accuracy, recall, and precision;

[0039] Collect feedback information on model detection results and perform continuous model optimization and parameter adjustment based on the feedback information.

[0040] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:

[0041] 1. The present invention combines the advantages of convolutional neural networks and recursive neural networks through the defect detection module, which can efficiently extract the spatial features and temporal features of the image, realize accurate defect detection and classification, and use the self-attention mechanism in the defect location module to further accurately locate the defect area. Specifically, convolutional neural networks are good at processing the spatial features of images, extracting the edges, textures and complex structural information of images through multiple convolutional layers, and capturing low-level and high-level features of images. Recursive neural networks excel in processing temporal features, capturing the temporal dependencies between feature vectors through a cyclic structure, so that the model can comprehensively consider the dynamic changes of features, thereby more accurately identifying and classifying defects.

[0042] 2. The present invention uses the self-attention mechanism in the defect location module to dynamically adjust the importance of each pixel by calculating the attention weight of each pixel in the feature map. The specific operation is to input the feature vectors extracted by the convolutional neural network and the recursive neural network into the self-attention model to generate a query matrix, a key matrix and a value matrix, and then determine the weight of each pixel by calculating the attention score. The introduction of the self-attention mechanism enables the model to focus on high-weight areas related to defects, thereby generating a weighted attention feature map. Based on the feature map, the location and range of the defect can be accurately identified.

[0043] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] In the figure:

[0046] Figure 1Flowchart of defect detection identification system.

[0047] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] The present invention will now be described in further detail with reference to the accompanying drawings.

[0049] See also Figure 1 As shown, in this embodiment, a production process defect detection and identification system based on deep learning is provided, including

[0050] An image data acquisition module, which is used to acquire image data of a product;

[0051] Data enhancement module, the data enhancement module is used to receive the image data generated by the image data acquisition module and preprocess the image data to generate high-quality synthetic defect images as the training set of the training model; by preprocessing and enhancing the image, high-quality synthetic defect images are generated, enriching the training data set. The advantage is that the diversity and robustness of model training are significantly improved. By adjusting the brightness, contrast, saturation and hue of the image, and adding random noise and various types of defects such as scratches, stains and cracks, various possible actual production environments and defect conditions are simulated. Alpha blending technology is used to generate complex scene images with multiple defects superimposed, further increasing the diversity of the data. Rich and diverse training data can improve the generalization ability of the model in different environments and defect types, making the trained model more stable and reliable in practical applications.

[0052] Defect detection module, the defect detection module uses trained convolutional neural networks and recurrent neural networks to extract image features and analyze temporal features; convolutional neural networks are good at processing spatial features of images and can extract low-level and high-level features in images, while recurrent neural networks can capture the temporal dependencies between these features. Combining the advantages of both, the system can process complex image data and perform in-depth analysis to achieve accurate identification of defects. Through the trained model, the module can quickly process large amounts of image data with a high degree of automation, reducing the errors and workload of manual detection.

[0053] Defect location module: The defect location module is used to receive the output results of the defect analysis module and use the self-attention mechanism to locate the defect position of the product; the defect location module can provide detailed defect location information to facilitate subsequent repair or processing. And cooperate with the self-attention mechanism to calculate the attention weight of each pixel in the feature map to generate a weighted attention feature map, thereby identifying possible defect locations. The identification of high-weight areas makes defect location more accurate. Use the boundary detection algorithm to determine the specific boundaries of the defect area and mark it on the original image to show the specific location and size of the defect.

[0054] Adaptive learning module, the adaptive learning module can automatically adjust parameters according to changes in the production environment, and continuously learn and optimize. By real-time monitoring of production environment data, such as temperature, humidity, light, etc., the system can dynamically adjust the parameters of the monitoring model to adapt it to current environmental changes. The mechanism of automatic adjustment and continuous learning enables the system to continuously optimize its own performance and reduce detection errors caused by environmental changes. In addition, collecting and analyzing feedback data from the test results can further improve the model and improve the accuracy and reliability of detection.

[0055] In this embodiment, when acquiring image data of a product, the image data acquisition module uses a camera to capture an image of the product and transmits the captured image data to the processing unit.

[0056] In this embodiment, the steps of the data enhancement module for receiving the image data generated by the image data acquisition module and preprocessing the image data to generate a high-quality synthetic defect image are as follows:

[0057] The preprocessed image data is received from the image data acquisition module and the brightness, contrast, saturation and hue of the image are obtained by adjusting the formula I′=α·I+β, where I is the original image, I′ is the adjusted image, and α and β are brightness and contrast adjustment parameters; by adjusting the brightness, contrast, saturation and hue of the image, the quality and consistency of the image data can be significantly improved, making it more suitable for subsequent defect detection and analysis. The adjustment of brightness and contrast helps to improve the visual effect of the image, making the details in the image clearer and easier to be recognized by the detection model. The adjustment of saturation and hue can enhance the color expression of the image and simulate the image effects under different lighting and shooting conditions. This preprocessing operation can not only improve the diversity of image data, but also effectively reduce the detection errors caused by changes in shooting conditions, ensuring the robustness and stability of the detection model in different environments.

[0058] Adding random noise and defects to images, such as scratches, stains, and cracks, can greatly enrich the diversity of the training data set, allowing the model to see more diverse image data during training. The addition of noise simulates various noise interference situations that may occur in the real production environment, and improves the robustness of the model in a noisy environment. Artificially added defects, such as scratches, stains, and cracks, directly simulate various defect situations that may occur in actual products, allowing the model to more accurately identify these defects in actual applications. Image data generated in this way can effectively improve the generalization ability of the model, enabling it to more reliably detect and identify various types of defects in actual production.

[0059] By formula Generate images with multiple defects superimposed, simulate complex scenes, and provide high-quality data for training models. final is the final generated multi-defect superposition image, α i is the transparency of the ith defect, is the i-th defect image, I base is the image after adjusting the brightness and contrast, and n is the number of defect types; by generating images with multiple defects superimposed, the data enhancement module can simulate a more complex and realistic production environment, thereby providing high-quality training data for the training model. This method can effectively reproduce the multiple defect superposition situations that may occur in the actual production of the product, so that the model can still accurately detect defects in complex scenes. The generation of multiple defect superimposed images not only increases the complexity of the training data, but also helps the model learn how to make correct judgments when multiple defects coexist, thereby improving the robustness and accuracy of the model.

[0060] Aggregating the images generated by all processing steps ensures the comprehensiveness and diversity of the training data. This process can integrate the image data generated by different processing steps to form a rich training data set, allowing the model to be exposed to various types of image data during the training process. By aggregating these image data, the generalization ability of the model can be effectively improved.

[0061] In this embodiment, the defect detection module uses the trained convolutional neural network and recursive neural network to extract image features and analyze time series features in the following steps:

[0062] The preprocessed image data is obtained from the image data acquisition module, and the spatial features of the image are extracted using a convolutional neural network. Using a convolutional neural network to extract the spatial features of the image can fully utilize the powerful capabilities of the convolutional neural network in image processing and feature extraction. The convolution layer can capture the low-level and high-level features of the image by extracting the edge, texture, and complex structural information in the image through the convolution kernel. This processing method can significantly improve the model's ability to understand and analyze image data, making defect detection more accurate and efficient. At the same time, the characteristics of the convolutional neural network make the model more robust to transformations such as rotation, scaling, and translation of the image, enhancing the stability and reliability of defect detection.

[0063] Using recurrent neural networks to The extracted features are used for time series analysis, and the outputs of convolutional neural networks and recurrent neural networks are combined for defect recognition and classification; recurrent neural networks are good at processing sequence data and can capture the temporal dependencies between features. By inputting the spatial features extracted by the convolutional neural network into the recurrent neural network for time series analysis, the temporal changes and patterns of features in the image can be identified, which is particularly important for defect detection. Combining the outputs of convolutional neural networks and recurrent neural networks, a comprehensive analysis of spatial and temporal features can be achieved, thereby improving the accuracy of defect recognition and classification. Recurrent neural networks can memorize and process long time series information, allowing the model to better understand the changes in temporal features in the image, which helps to identify complex defect situations.

[0064] By mapping the timing features to different defect categories and calculating the confidence score of each defect category through the formula, the type of defect can be accurately determined. Using the above formula for calculation can effectively evaluate the possibility of each defect category and ensure the reliability and accuracy of the detection results. The calculation of the confidence score is based on the output characteristics of the model, which can reflect the degree of judgment of the model on each category.

[0065] Reverse mapping the identified defect categories and corresponding features to the original image and marking the defect areas. Reverse mapping the identified defect categories and corresponding features to the original image and marking the defect areas can intuitively display the location and nature of the defects. This process not only helps to confirm the accuracy of the test results, but also provides clear guidance for subsequent defect repair and processing. By marking the defect areas on the original image, quality control personnel can quickly locate and identify defects, improving the efficiency of quality management in the production process.

[0066] In this embodiment, the convolutional neural network includes an input layer, a multi-layer convolutional layer, an activation layer, a pooling layer, and a fully connected layer. The steps of using the convolutional neural network to extract the spatial features of the image are as follows:

[0067] The image I is input into the convolutional neural network model and multiple convolutional layers are used to extract the low-level and high-level features of the image, which are expressed as: Among them, F l is the feature map of the lth layer, φ is the activation function ReLU, K l is the number of convolution kernels in layer l, W lk is the weight of the kth convolution kernel in the lth layer, F l-1 is the input feature map of the l-1th layer, b lk is the bias of the kth convolution kernel in the lth layer, and * represents the convolution operation. Through the superposition of multiple convolutional layers, the convolutional neural network can extract and integrate the rich information in the image layer by layer, so as to understand the image content more comprehensively. The introduction of the activation function ReLU increases the nonlinear ability of the network, allowing the model to better fit complex image features. In short, the use of convolutional layers makes the model highly efficient and accurate in feature extraction.

[0068] The ReLU activation function is used to increase the nonlinearity of the network, and the pooling layer is used to reduce the dimension of the feature map, reduce the amount of calculation, and retain important features. The extracted features are further processed through the fully connected layer to output a fixed-length feature vector, which is expressed as: f is the output fixed-length feature vector, n is the dimension of the flattened feature map, and w i is the weight of the fully connected layer, p i is the flattened output feature of the pooling layer, and b is the bias of the fully connected layer. The ReLU function introduces nonlinearity by changing the negative input values ​​to zero while keeping the positive values ​​unchanged. This nonlinear characteristic enables the model to learn and represent more complex data patterns, enhancing the network's expressive power. Compared with traditional activation functions (such as sigmoid or tanh), ReLU has the advantages of simple calculation, fast convergence speed, and reduced gradient vanishing problem. In addition, the ReLU function can also sparse the activation of the neural network, improving the training efficiency and robustness of the model.

[0069] In this embodiment, the steps of using a recursive neural network to perform time series analysis on the extracted features are:

[0070] The feature vectors extracted by the convolutional neural network are input into the recurrent neural network in the form of time series. The recurrent neural network is used to process the input sequence to capture the temporal dependency between the feature vectors. The expression is: t =φ(W h h t-1 +W xf t +b h ), where ht is the hidden state at the tth time step, W h is the weight matrix of the hidden state, W x is the weight matrix of the input feature vector, b h is the bias of the hidden state; the recurrent neural network saves and updates the timing information through its internal state. Finally, the output layer of the recurrent neural network generates the final timing feature representation. Inputting the feature vector extracted by the convolutional neural network into the recurrent neural network in the form of a time series can make full use of the advantages of the recurrent neural network in processing sequence data. Convolutional neural networks are good at extracting spatial features of images, while recurrent neural networks can capture the temporal dependencies between feature vectors. Combining these two networks can achieve a deeper understanding of image data. The recurrent neural network can save and update the timing information through its internal state mechanism, thereby effectively processing long time series data. The hidden state is updated at each time step, which not only retains the previous order information but also incorporates the information of the current step. This recursive update mechanism enables the recurrent neural network to have the ability to remember and process complex temporal dependencies. For defect detection, this means that the model can not only use the information of the current frame, but also integrate the context of the previous frame to make a more accurate judgment.

[0071] In this embodiment, the defect location module is used to receive the results output by the defect analysis module and use the self-attention mechanism to locate the defect position of the product:

[0072] Receive the defect detection results output by the defect analysis module and extract feature maps from the original image or the intermediate layer, and configure the number of attention heads and embedding dimensions of the self-attention model; by extracting feature maps from the intermediate layer, more image details and contextual information can be obtained, which is crucial for accurately locating defect areas. Configuring the number of attention heads and embedding dimensions of the self-attention model allows the model to process complex feature maps more flexibly and accurately. Increasing the number of attention heads can capture more diverse feature relationships, while the setting of the embedding dimension affects the granularity of feature representation.

[0073] By formula Calculate the attention weight of each pixel in the feature map and generate a weighted attention feature map based on the attention weight. Identify the defect location based on the high-weight area in the attention feature map, where Q is the query matrix, K is the key matrix, V is the value matrix, and d kis the dimension of the key vector; by calculating the attention weight of each pixel in the feature map, the important areas in the image can be accurately identified. The self-attention mechanism dynamically adjusts the importance of each pixel through the relationship between the query matrix, the key matrix, and the value matrix, so that the model can pay more attention to those areas related to defects. This weighted operation can not only improve the expressiveness of the feature map, but also accurately locate the defect position through the high-weight area.

[0074] Use boundary detection algorithm to determine the specific boundary of defect area and mark the defect area on the original image to show the specific location and size of the defect;

[0075] Summarize defect category, location, size and confidence information to form a complete positioning result. The boundary detection algorithm can accurately determine the specific boundaries of the defect area, providing a clear and definite basis for defect positioning. The boundary detection algorithm can effectively extract significant edges in the image, helping the model to accurately identify the outline and range of the defect.

[0076] In this embodiment, the adaptive learning module can automatically adjust parameters according to changes in the production environment. The steps of continuous learning and optimization are:

[0077] Collecting data and defect detection results of the production environment, and detecting changing trends and anomalies in the environmental data; collecting data and defect detection results of the production environment is the basis of the adaptive learning module. This process can obtain various environmental parameters on the production line in real time, such as temperature, humidity, light intensity, and equipment status, so as to fully understand the dynamic changes of the production environment. By continuously monitoring and analyzing these data, changing trends and anomalies in the environment can be discovered in a timely manner. For example, a sudden increase in temperature or abnormal vibration of equipment may indicate potential problems in the production process. A detailed analysis of environmental data helps to identify environmental factors that affect the accuracy of defect detection, and then take targeted measures to adjust and optimize.

[0078] Based on the extracted features and historical data, random forests are used to generate parameter adjustment strategies to determine how to adjust the parameters of the detection model; through the analysis of historical data, random forests can identify which environmental factors have the greatest impact on the detection results and generate corresponding parameter adjustment strategies. This method can ensure the rationality and effectiveness of model parameter adjustment and avoid the instability caused by blind adjustment.

[0079] Based on the latest environmental data and defect detection results, update the training data set and use the validation data set and actual production data to evaluate the detection accuracy, recall rate, and precision of the model; introduce the validation data set and actual production data to evaluate the detection accuracy, recall rate, and precision of the model, which can comprehensively measure the performance of the model. Update the training data set so that the model can learn and adapt to the current environmental changes and avoid the decline in detection accuracy caused by environmental changes.

[0080] Collect feedback information on model detection results and perform continuous model optimization and parameter adjustment based on the feedback information.

[0081] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.

Claims

1. A production process defect detection and identification system based on deep learning, characterized in that: include: An image data acquisition module, which is used to acquire image data of a product; A data enhancement module, which is used to receive the image data generated by the image data acquisition module and pre-process the image data to generate a high-quality synthetic defect image as a training set for the training model; Defect detection module, which uses trained convolutional neural networks and recurrent neural networks to extract image features and analyze time series features; Defect localization module: The defect localization module is used to receive the output results of the defect analysis module and use the self-attention mechanism to locate the defect position of the product; Adaptive learning module: The adaptive learning module can automatically adjust parameters according to changes in the production environment, and continuously learn and optimize.

2. According to the deep learning-based production process defect detection and identification system of claim 1, it is characterized in that: When acquiring image data of a product, the image data acquisition module uses a camera to capture an image of the product and transmits the captured image data to the processing unit.

3. The production process defect detection and identification system based on deep learning according to claim 1 is characterized in that: The steps of the data enhancement module for receiving the image data generated by the image data acquisition module and preprocessing the image data to generate a high-quality synthetic defect image are as follows: Receive the preprocessed image data from the image data acquisition module and adjust the brightness, contrast, saturation and hue of the image by the formula I′=α·I+β, where I is the original image, I′ is the adjusted image, and α and β are brightness and contrast adjustment parameters; Add random noise and defects to the image, including scratches, stains, and cracks; By formula Generate images with multiple defects superimposed, simulate complex scenes, and provide high-quality data for training models. final is the final multi-defect superposition image, α i is the transparency of the ith defect, is the i-th defect image, I base is the image after adjusting brightness and contrast, n is the number of defect types; Sum the images generated by all processing steps.

4. The production process defect detection and identification system based on deep learning according to claim 1 is characterized in that: The defect detection module uses the trained convolutional neural network and recurrent neural network to extract image features and analyze time series features as follows: Obtain preprocessed image data from the image data acquisition module, and use a convolutional neural network to extract spatial features of the image; Using recurrent neural networks to The features extracted are used for time series analysis, and the outputs of convolutional neural networks and recurrent neural networks are combined for defect recognition and classification. Map the timing features to different defect categories and calculate the confidence score of each defect category through the formula to determine the defect type; The identified defect categories and corresponding features are reversely mapped to the original image to mark the defective areas.

5. The production process defect detection and identification system based on deep learning according to claim 1 is characterized in that: The convolutional neural network includes an input layer, a multi-layer convolutional layer, an activation layer, a pooling layer, and a fully connected layer. The steps for extracting the spatial features of an image using a convolutional neural network are as follows: The image I is input into the convolutional neural network model and multiple convolutional layers are used to extract the low-level and high-level features of the image, which are expressed as: Among them, F l is the feature map of the lth layer, φ is the activation function ReLU, K l is the number of convolution kernels in layer l, W lk is the weight of the kth convolution kernel in the lth layer, F l-1 is the input feature map of the l-1th layer, b lk is the bias of the kth convolution kernel in the lth layer, and * represents the convolution operation; The ReLU activation function is used to increase the nonlinearity of the network, and the pooling layer is used to reduce the dimension of the feature map, reduce the amount of calculation, and retain important features. The extracted features are further processed through the fully connected layer to output a fixed-length feature vector, which is expressed as: f is the output fixed-length feature vector, n is the dimension of the flattened feature map, and w i is the weight of the fully connected layer, p i is the flattened output feature of the pooling layer, and b is the bias of the fully connected layer.

6. The production process defect detection and identification system based on deep learning according to claim 1 is characterized in that: The steps for performing time series analysis on the extracted features using a recurrent neural network are: The feature vectors extracted by the convolutional neural network are input into the recurrent neural network in the form of time series. The recurrent neural network is used to process the input sequence to capture the temporal dependency between the feature vectors. The expression is: t =φ(W h h t-1 +W x f t +b h ), where ht is the hidden state at the tth time step, W h is the weight matrix of the hidden state, W x is the weight matrix of the input feature vector, b h is the bias of the hidden state; the recurrent neural network saves and updates the timing information through its internal state. Finally, the output layer of the recurrent neural network generates the final timing feature representation.

7. The production process defect detection and identification system based on deep learning according to claim 1 is characterized in that: The defect location module is used to receive the output results of the defect analysis module and use the self-attention mechanism to locate the defect position of the product: Receive the defect detection results output by the defect analysis module and extract feature maps from the original image or intermediate layer, and configure the number of attention heads and embedding dimension of the self-attention model; By formula Calculate the attention weight of each pixel in the feature map and generate a weighted attention feature map based on the attention weight. Identify the defect location based on the high-weight area in the attention feature map, where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key vector; Use boundary detection algorithm to determine the specific boundary of defect area and mark the defect area on the original image to show the specific location and size of the defect; Summarize defect category, location, size and confidence information to form a complete positioning result.

8. The production process defect detection and identification system based on deep learning according to claim 1 is characterized in that: The adaptive learning module can automatically adjust parameters according to changes in the production environment. The steps for continuous learning and optimization are: Collect data and defect detection results from the production environment, and detect changing trends and anomalies in the environmental data; Based on the extracted features and historical data, a parameter adjustment strategy is generated using random forest to determine how the parameters of the detection model should be adjusted; Based on the latest environmental data and defect detection results, update the training dataset and use the validation dataset and actual production data to evaluate the model's detection accuracy, recall, and precision; Collect feedback information on model detection results and perform continuous model optimization and parameter adjustment based on the feedback information.