An edge-computing-based production line visual defect detection method and system

Abstract

The application relates to the technical field of visual defect detection, and discloses a production line visual defect detection method and system based on edge computing. The system comprises a multi-angle image acquisition module, an edge-side preliminary detection module, a cloud-edge data transmission module, a cloud-side deep processing and optimization module, and a high-precision detection execution module. A preliminary defect detection model is deployed and run on an edge computing device to perform real-time preliminary defect detection on preprocessed visual image data, and generated preliminary defect detection data and image uncertainty region data are uploaded asynchronously, so that the amount of data uploaded to a cloud server can be reduced, network transmission bandwidth pressure and overall calculation delay can be reduced, the demand of a high-speed production line for real-time detection and response can be met, and the risk of defect product escape can be reduced.

Description

Technical Field

[0001] This invention relates to the field of visual defect detection technology, specifically to a method and system for visual defect detection on production lines based on edge computing. Background Technology

[0002] There is a significant demand for defect detection in the industrial sector. With the development of machine vision image processing technology, in some high-speed, large-format, and high-precision inspection projects, defect detection work that was traditionally performed by humans has begun to be replaced by machine vision. The use of visual inspection equipment to replace human eyes in judgment and measurement improves the flexibility and automation of production.

[0003] Currently, in the field of visual defect detection on production lines, traditional solutions typically rely on detection models deployed on local industrial control computers and centralized servers. Due to the large number of parameters and computational complexity of the detection models, computational and transmission delays occur when processing high-resolution, multi-angle production line image streams, making it difficult to meet the real-time detection and response requirements of high-speed production lines, resulting in defective products escaping and a decrease in production line cycle time.

[0004] Therefore, a production line visual defect detection method and system based on edge computing is proposed to solve the above problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a production line visual defect detection method and system based on edge computing, which solves the problem mentioned in the background that it is difficult to meet the real-time detection and response requirements of high-speed production lines, resulting in defective products escaping and production line cycle time decreasing.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method and system for visual defect detection on a production line based on edge computing, the method comprising the following steps: S1. Collect multi-angle visual image data of products on the production line; S2. Perform standardized preprocessing on the multi-angle visual image data to generate preprocessed visual image data; S3. Input the visual image data into the primary defect detection model deployed on the edge computing device, perform detection, and generate primary defect detection data and image uncertainty region data; S4. Upload the preprocessed visual image data, the primary defect detection data, and the image uncertainty region data to the cloud server; S5. Based on the image uncertainty region data, feature extraction is performed on the corresponding region in the visual image data in the cloud server to generate depth feature data; S6. Based on the primary defect detection data and the deep feature data, incremental learning and parameter optimization are performed on the benchmark defect detection model in the cloud server to generate an optimized defect detection model and model optimization strategy data. S7. Send the optimized defect detection model and the model optimization strategy data to the edge computing device to update the parameters of the primary defect detection model; S8. Based on the updated primary defect detection model, the visual image data is detected to generate final defect detection result data. S9. Based on the final defect detection results data, generate a product defect classification, location report, and control instructions for the production line.

[0007] Preferably, the step S1 of acquiring multi-angle visual image data of production line products includes the following steps: S11. Deploy multiple industrial cameras and light source systems at preset coordinate points on the production line to form a multi-view image acquisition array; S12. Through the multi-view image acquisition array, when the product flows through the inspection station, the acquisition of multi-angle visual image data of the product surface under different preset lighting conditions is triggered simultaneously. The multi-angle visual image data includes visual image data of the top view, left view, and right view of the product. S13. Add timestamps, production line station identifiers, and product serial number identifiers to the collected multi-angle visual image data to form a set of original visual image data with multi-dimensional association information.

[0008] Preferably, the standardization preprocessing in S2 includes the following steps: S21. Obtain the original visual image data set; S22. Perform preprocessing operations such as noise filtering, illumination non-uniformity correction, and image geometric distortion correction on each frame of the original visual image data set in sequence. S23. Scale the visual image data that has completed the preprocessing operation to a preset resolution and perform pixel value normalization processing to generate the preprocessed visual image data.

[0009] Preferably, the generation of primary defect detection data and image uncertainty region data in step S3 includes the following steps: S31. Deploy and load the primary defect detection model on an edge computing device. The model is a convolutional neural network model that employs pruning and quantization processing. S32. Input the preprocessed visual image data into the primary defect detection model and perform forward inference calculation; S33. The primary defect detection model outputs the primary defect detection data, which includes the defect existence confidence level, the preliminary defect category probability distribution, and the preliminary defect bounding box coordinates. S34. Simultaneously, calculate the uncertainty metric value of each pixel position in the preprocessed visual image data during the model inference process, and mark the regions with uncertainty metric values ​​higher than a preset threshold as the image uncertainty region data, wherein the uncertainty region data includes region coordinates and uncertainty score.

[0010] Preferably, the upload to the cloud server in step S4 includes the following steps: S41. In the edge computing device, the preprocessed visual image data, the primary defect detection data, and the image uncertainty region data are packaged into a data packet; S42. The data packet is asynchronously uploaded to the designated storage area of ​​the cloud server through an encrypted transmission channel; S43. After receiving the data packet, the cloud server parses and establishes an index association relationship between the preprocessed visual image data, the primary defect detection data, and the image uncertainty region data, and stores it in the cloud database.

[0011] Preferably, feature extraction in step S5 includes the following steps: S51. From the cloud database, based on the coordinates indicated by the image uncertainty region data, extract the corresponding image region block from the preprocessed visual image data as the image block to be analyzed; S52. Call the feature extraction network in the cloud server to perform multi-level, multi-scale feature extraction on the image block to be analyzed; S53. The extracted features are fused to generate the deep feature data, and the data is associated with and stored with the corresponding primary defect detection data.

[0012] Preferably, the incremental learning and parameter optimization in S6 includes the following steps: S61. Store a baseline defect detection model in a cloud server; S62. Using the associated stored deep feature data as input and the manually verified defect annotation data as a supervision signal, construct a model optimization training sample set; S63. Use the model to optimize the training sample set, perform online incremental learning on the benchmark defect detection model, adjust the weight parameters of the benchmark defect detection model, and adopt a preset incremental learning algorithm in the learning process. S64. After each incremental learning is completed, evaluate the performance indicators of the optimized model and generate the model optimization strategy data describing the parameters of this optimization process. The model optimization strategy data includes the sample feature fingerprints used for optimization, the changes in model parameters, and the performance evaluation indicators. S65. Save the model whose optimized performance evaluation index reaches the preset threshold as the optimized defect detection model.

[0013] Preferably, updating the parameters of the primary defect detection model in step S7 includes the following steps: S71. The cloud server generates a model conversion scheme for deployment on edge computing devices based on the structure of the optimized defect detection model and the model optimization strategy data. S72. Based on the model conversion scheme, perform model distillation and structured pruning operations on the optimized defect detection model, and perform knowledge refinement operations on the pruned model to obtain an intermediate model file oriented towards the computing power constraints of edge computing devices. S73. Package the intermediate model file and the model optimization strategy data into an update file; S74. After receiving the updated file, the edge computing device, during the detection interval and without interrupting the detection process, stops the original primary defect detection model, loads the intermediate model file as the new primary defect detection model, completes the model update, and loads the model optimization strategy data.

[0014] Preferably, generating the final defect detection result data in step S8 includes the following steps: S81. The updated primary defect detection model detects the subsequently incoming preprocessed visual image data. S82. The model output includes the final defect detection result data, including defect category, defect location bounding box, and defect severity score. S83. Bind the final defect detection result data with the corresponding product serial number and timestamp to form a structured detection record.

[0015] Preferably, the system includes: The multi-angle image acquisition module acquires multi-angle visual image data of products on the production line through an image acquisition array, and outputs pre-processed visual image data after processing by the pre-processing unit. The edge-side preliminary detection module, deployed on the edge computing device, processes the preprocessed visual image data through a primary defect detection model and outputs primary defect detection data and image uncertainty region data. The cloud-edge data transmission module uploads the preprocessed visual image data, primary defect detection data, and image uncertainty region data to the cloud server through the data transmission channel, and receives model update data from the cloud server. The cloud-based deep processing and optimization module, deployed on a cloud server, processes the image data corresponding to the uncertain regions of the image through the feature enhancement extraction unit to generate deep feature data, and uses the deep feature data and primary defect detection data to incrementally learn the benchmark defect detection model through the model training unit, outputting the optimized defect detection model and model optimization strategy data. The high-precision detection execution module, deployed on the updated edge computing device, processes the pre-processed visual image data through the updated primary defect detection model to generate the final defect detection result data. The report and control unit then generates a defect report and control commands based on this result data.

[0016] Compared with existing technologies, this invention provides a production line visual defect detection method and system based on edge computing, which has the following beneficial effects: 1. In this invention, by deploying and running a primary defect detection model on an edge computing device, real-time preliminary defect detection is performed on the preprocessed visual image data. The generated primary defect detection data and image uncertainty area data are uploaded asynchronously, which can reduce the amount of data that needs to be uploaded to the cloud server, reduce network transmission bandwidth pressure and overall computing latency, thereby meeting the needs of high-speed production lines for real-time detection and response, and reducing the risk of defective products escaping.

[0017] 2. In this invention, deep feature data is generated by performing feature enhancement extraction on the corresponding region based on image uncertainty region data in the cloud server. Based on this, the baseline defect detection model is incrementally learned online with primary defect detection data to generate an optimized defect detection model and distribute it to the updated edge computing device. This can utilize the centralized storage and computing resources of the cloud server to continuously optimize the detection algorithm, improve the model's discrimination ability for difficult and uncertain samples, thereby improving the accuracy and recall of defect detection and reducing false detections and false negatives.

[0018] 3. In this invention, by constructing a closed-loop optimization process from real-time edge detection, incremental learning in the cloud to dynamic model deployment, the primary defect detection model deployed on the edge can continuously and adaptively optimize based on actual production data, without the need for frequent manual re-labeling of data and offline training and deployment. This improves the defect detection system's adaptability and generalization ability to new products and new defect types on the production line, and reduces model maintenance costs and update cycles. Attached Figure Description

[0019] Figure 1This is a flowchart of a production line visual defect detection method based on edge computing according to the present invention; Figure 2 This is a schematic diagram of the architecture of a production line visual defect detection system based on edge computing according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] For specific implementation examples, please refer to: Figures 1-2 A method and system for visual defect detection on a production line based on edge computing, the method comprising the following steps: S1. Collect multi-angle visual image data of products on the production line; S2. Standardize and preprocess the multi-angle visual image data to generate preprocessed visual image data; S3. Input the visual image data into the primary defect detection model deployed on the edge computing device, perform detection, and generate primary defect detection data and image uncertainty area data; S4. Upload the preprocessed visual image data, primary defect detection data, and image uncertainty area data to the cloud server; S5. Based on image uncertainty region data, feature extraction is performed on the corresponding region in the visual image data in the cloud server to generate depth feature data; S6. Based on primary defect detection data and deep feature data, incremental learning and parameter optimization are performed on the benchmark defect detection model in the cloud server to generate an optimized defect detection model and model optimization strategy data. S7. Send the optimized defect detection model and model optimization strategy data to the edge computing device to update the parameters of the primary defect detection model; S8. Based on the updated primary defect detection model, the visual image data is detected to generate the final defect detection result data. S9. Based on the final defect detection results, generate a product defect classification and location report and control instructions for the production line.

[0022] The steps involved in acquiring multi-angle visual image data of production line products in S1 are as follows: S11. Deploy multiple industrial cameras and light source systems at preset coordinate points on the production line to form a multi-view image acquisition array; S12. Through a multi-view image acquisition array, when the product flows through the inspection station, the acquisition of multi-angle visual image data of the product surface under different preset lighting conditions is triggered simultaneously. The multi-angle visual image data includes visual image data of the top view, left view, and right view of the product. S13. Add timestamps, production line station identifiers, and product serial number identifiers to the collected multi-angle visual image data to form a set of original visual image data with multi-dimensional correlation information.

[0023] The standardization preprocessing in S2 includes the following steps: S21. Obtain the original visual image data set; S22. Perform preprocessing operations on each frame of the original visual image dataset, including noise filtering, illumination non-uniformity correction, and image geometric distortion correction, in sequence, including the following steps: S221. Noise filtering operation: Perform spatial domain smoothing on each frame of visual image to suppress high-frequency random noise introduced during image acquisition. S222, Illumination non-uniformity correction operation: Analyze the image after noise filtering, adjust the gray value distribution of each pixel in the image to achieve uniformity of the overall illumination intensity of the image; S223, Image geometric distortion correction operation: Based on the pre-acquired camera calibration parameters, perform inverse geometric transformation mapping on the image to correct the image geometric distortion caused by the nonlinear characteristics of the optical lens; S23. Scale the preprocessed visual image data to a preset resolution and perform pixel value normalization to generate preprocessed visual image data.

[0024] The steps involved in generating primary defect detection data and image uncertainty region data in S3 are as follows: S31. Deploy and load a primary defect detection model on an edge computing device. The model is a convolutional neural network model with pruning and quantization processing. S32. Input the preprocessed visual image data into the primary defect detection model and perform forward inference calculations, including the following steps: S321. The preprocessed visual image data is loaded as an input tensor into the input layer of the primary defect detection model; S322. The primary defect detection model calculates the input tensor sequentially through its convolutional layers, activation function layers, and pooling layers according to a preset network layer order, where: Convolutional layer computation: This layer performs convolution operations between the filter and the input feature map and adds a bias to generate the output feature map. Layer Each feature map at location value The calculation is as follows: ; in, To output the two-dimensional spatial coordinates on the feature map, This represents the two-dimensional spatial coordinate offset within the convolution filter. The number of input feature maps, and For the height and width of the filter, To connect the first Layer The first feature map and the first feature map of this layer The filter of each feature map at the location The weight, For the first Layer Each output feature map For the first Layer The bias of each output feature map; Activation function layer calculation: Applying an activation function to the output of the convolutional layer generates activation values. ; in, For the first Layer Each feature map in The input value for the location. This represents the activation function. This indicates that after processing by the activation function, the th Layer Each feature map in The final output value of the position; Geomorphic layer calculation: The input feature map is downsampled. Taking max pooling as an example, in... Within the pooling window, the output value is calculated as follows: ; in, express No. The pixel values ​​at corresponding positions in each feature map This indicates that after max pooling, the current i-th... Layer Each feature map in The output value of the position, This is the step size for the pooling operation. The size of the pooling window; S323. After sequential calculation through the network layers, an output tensor corresponding to the visual image data is obtained in the output layer of the model. This output tensor contains the original logical values ​​used to generate primary defect detection data. S33. The primary defect detection model outputs primary defect detection data, which includes defect existence confidence, preliminary defect category probability distribution, and preliminary defect bounding box coordinates. S34. Simultaneously, calculate the uncertainty metric value of each pixel position in the preprocessed visual image data during the model inference process, and mark the regions with uncertainty metric values ​​higher than a preset threshold as image uncertainty region data. The uncertainty region data includes region coordinates and uncertainty scores. In the forward inference process of the primary defect detection model, the following steps are included: Execute the primary defect detection model... Second Monte Carlo sampling, obtained Different model outputs; S342. For each pixel position, according to The model output is used to calculate the variance of the probability that the pixel is predicted to be of the defect category. This variance is used as a measure of the uncertainty at the pixel location and is calculated using the following formula: ; in, The pixel coordinates in the image. Indicates the coordinates in the image are The uncertainty metric of the pixel. Indicates the first During the Monte Carlo sampling, the probability that the model predicts the pixel to be of the defect category is... The total number of samples taken in Monte Carlo. express The mean of the sampling probabilities.

[0025] Uploading to a cloud server in S4 includes the following steps: S41. In the edge computing device, the preprocessed visual image data, primary defect detection data and image uncertainty region data are packaged into a data package; S42. Asynchronously upload the data packet to the designated storage area of ​​the cloud server through an encrypted transmission channel, including the following steps: S421. The edge computing device opens an upload queue in the local cache and places the data packets to be uploaded into the upload queue. S422. Edge computing devices continuously monitor the network bandwidth status between themselves and cloud servers; S423. When the network bandwidth is detected to be higher than the preset threshold, data packets are retrieved from the upload queue in first-in-first-out order and uploaded to the cloud server through the encrypted transmission channel. S424. After receiving the data packet, the cloud server returns a reception confirmation signal to the edge computing device; S43. After receiving the data packet, the cloud server parses and establishes an index association between the preprocessed visual image data, primary defect detection data, and image uncertainty area data, and stores it in the cloud database.

[0026] Feature extraction in S5 includes the following steps: S51. From the cloud database, based on the coordinates indicated by the image uncertainty region data, extract the corresponding image region block from the preprocessed visual image data as the image block to be analyzed. S52. Call the feature extraction network in the cloud server to perform multi-level, multi-scale feature extraction on the image patch to be analyzed, including the following steps: S521. Input the image patch to be analyzed into the feature extraction network; S522. The feature extraction network extracts shallow feature maps, middle feature maps and deep feature maps of the image patch to be analyzed in sequence through its convolutional layers. S523. Perform upsampling and downsampling operations on the shallow, middle, and deep feature maps respectively, adjusting them to the same spatial size. Specifically, this includes: Shallow feature map upsampling: For shallow feature maps, since their spatial resolution is greater than the resolution corresponding to the target size, we directly perform upsampling operations to enlarge their spatial size. Mid-layer feature map size preservation: For mid-layer feature maps, since their current spatial size is equal to the target size, their spatial size is kept unchanged; Deep feature map downsampling: For deep feature maps, since their spatial resolution is smaller than the resolution corresponding to the target size, a downsampling operation is directly performed on them to reduce their spatial size; S524. After adjusting the shallow feature map, middle feature map and deep feature map to the same size, they are stitched together in the channel dimension to generate a fused feature map. S53. The extracted features are fused to generate deep feature data, which is then associated with and stored with the corresponding primary defect detection data.

[0027] Incremental learning and parameter optimization in S6 includes the following steps: S61. Store a baseline defect detection model in a cloud server; S62. Using the associated stored deep feature data as input and the manually verified defect annotation data as supervision signal, construct a model optimization training sample set; S63. Use the model to optimize the training sample set, perform online incremental learning on the benchmark defect detection model, adjust the weight parameters of the benchmark defect detection model, and the learning process adopts a preset incremental learning algorithm, including the following steps: S631. Store an experience replay buffer in the cloud server to store historical training samples; S632. Merge the currently constructed model optimization training sample set with a portion of historical training samples randomly sampled from the experience replay buffer to form a mixed training set; S633. Train the benchmark defect detection model using a hybrid training set to adjust its weight parameters. The training process includes minimizing the loss function. The process is defined as follows: ; in, As a benchmark defect detection model in parameters The total loss value below, This is the set of weight parameters for the benchmark defect detection model. This represents the total number of samples in the mixed training set. For the first Each sample in the current model parameters The loss function value under the following conditions The regularization coefficient is . For regularization terms of model parameters; S634. Store a portion of the samples in the current model optimization training sample set into the experience replay buffer. S635. When the storage capacity of the experience playback buffer reaches the upper limit threshold. At that time, the earliest stored sample is removed according to the first-in, first-out principle to maintain the stability of the buffer capacity; upper limit threshold Determined by the following formula: ; in, This represents the upper limit threshold of the experience playback buffer capacity. This indicates the preset review time window length. This represents the average sampling frequency of valid samples uploaded from the edge of the production line to the cloud for model optimization. S64. After each incremental learning iteration, evaluate the performance metrics of the optimized model and generate model optimization strategy data describing the parameters of this optimization process. The model optimization strategy data includes the sample feature fingerprints used for optimization, the changes in model parameters, and performance evaluation metrics, including the following steps: S641. Extract a batch of verification image data with confirmed defect annotations from the cloud database; S642. Use the optimized defect detection model after incremental learning to perform defect detection on the verification image data and obtain the prediction results. S643. Compare the prediction results with the confirmed defect annotations, and calculate the recall, precision, and F1 score of the model on the current validation set as performance indicators; wherein, the recall, precision, and F1 score are calculated using the following formulas: ; ; ; in, To achieve recall, For precision, For F1 score, This represents the number of true positives, i.e., the number of samples that the model correctly predicted as defects. This represents the number of false positives, that is, the number of samples that the model incorrectly predicted as defects. This indicates the number of false negatives, i.e., the number of samples that the model incorrectly predicted as defect-free. S65. Save the model whose optimized performance evaluation index reaches the preset threshold as an optimized defect detection model.

[0028] Updating the parameters of the primary defect detection model in S7 includes the following steps: S71. The cloud server generates a model conversion scheme for deployment on edge computing devices based on the structure and model optimization strategy data of the optimized defect detection model. S72. Based on the model conversion scheme, perform model distillation and structured pruning operations on the optimized defect detection model, and perform knowledge refinement operations on the pruned model to obtain an intermediate model file oriented towards the computing power constraints of edge computing devices. The model distillation operation for optimizing the defect detection model includes the following steps: The optimized defect detection model is used as the teacher model, and a student model with a simpler structure is initialized. Using the same batch of input data, forward propagation is performed through the teacher model and the student model respectively to obtain the soft label output of the teacher model and the prediction output of the student model; The KL divergence loss between the predicted output of the student model and the soft-label output is calculated using the following formula: ; in, This represents the knowledge distillation loss. The temperature parameter is the one used in the Softmax function. The total number of categories. and These represent the teacher model and student model respectively for the first... Output logical values ​​for each category. This represents the softmax function; The weight parameters of the student model are updated with the goal of minimizing the KL divergence loss. Performing structured pruning on the optimized defect detection model includes the following steps: Identify and optimize the weight parameters of convolutional layers in the defect detection model; Calculate the L1 norm of each filter in each convolutional layer, specifically for the _ ... Layer The calculation of each filter is as follows: ; in, Indicates the first The first in the layer The L1 norm of the filter weight tensor Input the number of channels. and The height and width of the filter space dimensions. For this filter in the input channel Spatial location The weight value at that location, This refers to the height and width indexes of the spatial coordinates inside the waveguide. Based on the preset pruning rate, remove the filters with the smallest L1 norm value and their corresponding output feature channels. Synchronously remove the input channels of the next layer connected to the removed filters in the pruned model; Perform knowledge refinement on the pruned model, including the following steps: Forward inference was performed on the pruned model using a small, labeled calibration dataset; Calculate the model loss based on the error between the inference results and the labeled data: ; in, This represents the total loss value of the model. Represents the standard cross-entropy loss. Represents the regularization loss term. Represents the regularization coefficient; The gradient descent algorithm is used to fine-tune the remaining weight parameters of the pruned model based on the loss. The fine-tuning process is calculated using the following backpropagation and parameter update formula: ; in, This represents the updated model weight parameters. This represents the model weight parameters before the update. Indicates the learning rate. For the total loss function with respect to parameters exist gradient at; S73. Package the intermediate model file and the model optimization strategy data into an update file; S74. After receiving the update file, the edge computing device, during the detection interval and without interrupting the detection process, stops the original primary defect detection model, loads the intermediate model file as the new primary defect detection model, completes the model update, and loads the model optimization strategy data.

[0029] The steps involved in generating the final defect detection result data in S8 are as follows: S81. The updated primary defect detection model detects subsequent incoming preprocessed visual image data. S82. The model outputs final defect detection results data, including defect category, defect location bounding box, and defect severity score; the defect severity score is determined through the following steps: S821. Based on the defect category in the final defect detection result data, query the preset defect category-basic severity mapping table to obtain the basic severity score; S822. Calculate the ratio of the defect area to the pre-defined area of ​​interest on the product surface based on the area of ​​the defect location bounding box. S823. The base severity score is weighted and summed with the proportion. The result is used as the defect severity score, specifically calculated using the following formula: ; in, This indicates the final score for the severity of the defect. This represents the baseline severity score obtained from the mapping table. This represents the area of ​​the defect bounding box. This indicates the area of ​​the pre-defined area of ​​interest on the product surface. and The preset weighting coefficients are used, and they satisfy the following conditions: ; S83. Bind the final defect detection result data with the corresponding product serial number and timestamp to form a structured detection record.

[0030] The system includes: The multi-angle image acquisition module acquires multi-angle visual image data of products on the production line through an image acquisition array, and outputs pre-processed visual image data after processing by the pre-processing unit. The edge-side preliminary detection module, deployed on the edge computing device, processes the preprocessed visual image data through the primary defect detection model and outputs primary defect detection data and image uncertainty area data. The cloud-edge data transmission module uploads preprocessed visual image data, primary defect detection data, and image uncertainty region data to the cloud server through the data transmission channel, and receives model update data from the cloud server. The cloud-based deep processing and optimization module, deployed on a cloud server, processes image data corresponding to uncertain regions of the image through a feature enhancement extraction unit to generate deep feature data. Then, through a model training unit, it incrementally learns the benchmark defect detection model using the deep feature data and primary defect detection data, and outputs an optimized defect detection model and model optimization strategy data. The high-precision detection execution module, deployed on the updated edge computing device, processes the pre-processed visual image data through the updated primary defect detection model to generate the final defect detection result data. The report and control unit then generates a defect report and control commands based on this result data.

[0031] The operation steps of this edge computing-based production line visual defect detection method and system are as follows: Step 1: Multi-angle image acquisition and standardization preprocessing: The system first deploys a multi-view image acquisition array consisting of multiple industrial cameras and light sources at predetermined locations on the production line. As products flow through the inspection station, the array is synchronously triggered, acquiring visual image data of the product surface from the top, left, and right sides under different lighting conditions. These raw images are timestamped and marked with station identifiers to form a raw visual image data set. Subsequently, the system sequentially performs noise filtering, illumination non-uniformity correction, and image geometric distortion correction on each frame in this set, then uniformly scales it to a preset resolution and normalizes pixel values, finally outputting standardized, pre-processed visual image data suitable for model processing.

[0032] Step 2: Real-time preliminary detection and key data screening at the edge: The preprocessed image data is input in real time into a primary defect detection model deployed on an edge computing device for forward inference computation. This model quickly outputs primary defect detection data, including the confidence level, preliminary category, and location of the defect. Simultaneously, the system calculates the uncertainty metric for each pixel in the image during the model's inference process and marks areas with uncertain metrics above a threshold as image uncertainty regions.

[0033] Step 3: Asynchronous Data Upload and Deep Feature Extraction in the Cloud The edge device packages the preprocessed image, initial detection results, and marked uncertainty region data, and asynchronously uploads them to the cloud server via an encrypted channel, uploading only critical data to save bandwidth. Upon receiving the data, the cloud server specifically targets the image patches corresponding to the uncertainty regions in the image, calling a more powerful feature extraction network to perform multi-level, multi-scale deep feature extraction, generating deep feature data rich in detail and semantic information, and storing it in association with the corresponding initial detection results.

[0034] Step 4: Cloud-based incremental learning and model optimization: A baseline defect detection model is stored on a cloud server. The system uses the associated stored deep feature data as input and manually verified defect annotations as supervision signals to construct a training sample set. Subsequently, an online incremental learning algorithm incorporating an experience replay buffer is used to train the baseline model with new samples, optimizing its weight parameters so that the model learns how to more accurately detect previously uncertain defect types. After learning is complete, the model performance is evaluated, and models that meet the performance targets are saved as optimized defect detection models, and the model optimization strategy data for this learning is recorded.

[0035] Step 5: Model Lightweighting and Dynamic Edge Updates Based on the optimized model's structure and optimization strategy, the cloud server transforms the large model into a lightweight model file that matches the computing power constraints of the edge computing devices through model distillation, structured pruning, and knowledge refinement operations. This model file, along with the optimization strategy data, is packaged into an update file and distributed to the corresponding edge computing devices on the production line. The edge devices deactivate the old model and load the new model during intermittent but uninterrupted detection, completing the parameter update of the primary defect detection model.

[0036] Step Six: Closed-Loop Optimization and High-Precision Detection The updated primary defect detection model improves detection accuracy and generalization ability when detecting subsequent product images at the edge, and outputs final defect detection results data containing defect category, location and severity scores. These results are used to generate defect reports and control instructions, and the whole process forms a closed loop of continuous optimization.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0038] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A production line visual defect detection method based on edge computing, characterized in that: The method includes the following steps: S1. Collect multi-angle visual image data of products on the production line; S2. Perform standardized preprocessing on the multi-angle visual image data to generate preprocessed visual image data; S3. Input the visual image data into the primary defect detection model deployed on the edge computing device, perform detection, and generate primary defect detection data and image uncertainty region data; S4. Upload the preprocessed visual image data, the primary defect detection data, and the image uncertainty region data to the cloud server; S5. Based on the image uncertainty region data, feature extraction is performed on the corresponding region in the visual image data in the cloud server to generate depth feature data; S6. Based on the primary defect detection data and the deep feature data, incremental learning and parameter optimization are performed on the benchmark defect detection model in the cloud server to generate an optimized defect detection model and model optimization strategy data. S7. Send the optimized defect detection model and the model optimization strategy data to the edge computing device to update the parameters of the primary defect detection model; S8. Based on the updated primary defect detection model, the visual image data is detected to generate final defect detection result data. S9. Based on the final defect detection results data, generate a product defect classification, location report, and control instructions for the production line.

2. The production line visual defect detection method based on edge computing according to claim 1, characterized in that: The step of collecting multi-angle visual image data of production line products in S1 includes the following steps: S11. Deploy multiple industrial cameras and light source systems at preset coordinate points on the production line to form a multi-view image acquisition array; S12. Through the multi-view image acquisition array, when the product flows through the inspection station, the acquisition of multi-angle visual image data of the product surface under different preset lighting conditions is triggered simultaneously. The multi-angle visual image data includes visual image data of the top view, left view, and right view of the product. S13. Add timestamps, production line station identifiers, and product serial number identifiers to the collected multi-angle visual image data to form a set of original visual image data with multi-dimensional association information.

3. The production line visual defect detection method based on edge computing according to claim 2, characterized in that: The standardization preprocessing in S2 includes the following steps: S21. Obtain the original visual image data set; S22. Perform preprocessing operations such as noise filtering, illumination non-uniformity correction, and image geometric distortion correction on each frame of the original visual image data set in sequence. S23. Scale the visual image data that has completed the preprocessing operation to a preset resolution and perform pixel value normalization processing to generate the preprocessed visual image data.

4. The production line visual defect detection method based on edge computing according to claim 3, characterized in that: The generation of primary defect detection data and image uncertainty region data in S3 includes the following steps: S31. Deploy and load the primary defect detection model on an edge computing device. The model is a convolutional neural network model that employs pruning and quantization processing. S32. Input the preprocessed visual image data into the primary defect detection model and perform forward inference calculation; S33. The primary defect detection model outputs the primary defect detection data, which includes the defect existence confidence level, the preliminary defect category probability distribution, and the preliminary defect bounding box coordinates. S34. Simultaneously, calculate the uncertainty metric value of each pixel position in the preprocessed visual image data during the model inference process, and mark the regions with uncertainty metric values ​​higher than a preset threshold as the image uncertainty region data, wherein the uncertainty region data includes region coordinates and uncertainty score.

5. The production line visual defect detection method based on edge computing according to claim 4, characterized in that: The upload to the cloud server in S4 includes the following steps: S41. In the edge computing device, the preprocessed visual image data, the primary defect detection data, and the image uncertainty region data are packaged into a data packet; S42. The data packet is asynchronously uploaded to the designated storage area of ​​the cloud server through an encrypted transmission channel; S43. After receiving the data packet, the cloud server parses and establishes an index association relationship between the preprocessed visual image data, the primary defect detection data, and the image uncertainty region data, and stores it in the cloud database.

6. The production line visual defect detection method based on edge computing according to claim 5, characterized in that: Feature extraction in step S5 includes the following steps: S51. From the cloud database, based on the coordinates indicated by the image uncertainty region data, extract the corresponding image region block from the preprocessed visual image data as the image block to be analyzed; S52. Call the feature extraction network in the cloud server to perform multi-level, multi-scale feature extraction on the image block to be analyzed; S53. The extracted features are fused to generate the deep feature data, and the data is associated with and stored with the corresponding primary defect detection data.

7. The production line visual defect detection method based on edge computing according to claim 6, characterized in that: The incremental learning and parameter optimization in S6 includes the following steps: S61. Store a baseline defect detection model in a cloud server; S62. Using the associated stored deep feature data as input and the manually verified defect annotation data as a supervision signal, construct a model optimization training sample set; S63. Use the model to optimize the training sample set, perform online incremental learning on the benchmark defect detection model, adjust the weight parameters of the benchmark defect detection model, and adopt a preset incremental learning algorithm in the learning process. S64. After each incremental learning is completed, evaluate the performance indicators of the optimized model and generate the model optimization strategy data describing the parameters of this optimization process. The model optimization strategy data includes the sample feature fingerprints used for optimization, the changes in model parameters, and the performance evaluation indicators. S65. Save the model whose optimized performance evaluation index reaches the preset threshold as the optimized defect detection model.

8. The production line visual defect detection method based on edge computing according to claim 7, characterized in that: The parameter update of the primary defect detection model in step S7 includes the following steps: S71. The cloud server generates a model conversion scheme for deployment on edge computing devices based on the structure of the optimized defect detection model and the model optimization strategy data. S72. Based on the model conversion scheme, perform model distillation and structured pruning operations on the optimized defect detection model, and perform knowledge refinement operations on the pruned model to obtain an intermediate model file oriented towards the computing power constraints of edge computing devices. S73. Package the intermediate model file and the model optimization strategy data into an update file; S74. After receiving the updated file, the edge computing device, during the detection interval and without interrupting the detection process, stops the original primary defect detection model, loads the intermediate model file as the new primary defect detection model, completes the model update, and loads the model optimization strategy data.

9. The production line visual defect detection method based on edge computing according to claim 8, characterized in that: The process of generating the final defect detection result data in S8 includes the following steps: S81. The updated primary defect detection model detects the subsequently incoming preprocessed visual image data. S82. The model output includes the final defect detection result data, including defect category, defect location bounding box, and defect severity score. S83. Bind the final defect detection result data with the corresponding product serial number and timestamp to form a structured detection record.

10. A production line visual defect detection system based on edge computing, used to implement the production line visual defect detection method based on edge computing as described in any one of claims 1 to 9, characterized in that, The system includes: The multi-angle image acquisition module acquires multi-angle visual image data of products on the production line through an image acquisition array, and outputs pre-processed visual image data after processing by the pre-processing unit. The edge-side preliminary detection module, deployed on the edge computing device, processes the preprocessed visual image data through a primary defect detection model and outputs primary defect detection data and image uncertainty region data. The cloud-edge data transmission module uploads the preprocessed visual image data, primary defect detection data, and image uncertainty region data to the cloud server through the data transmission channel, and receives model update data from the cloud server. The cloud-based deep processing and optimization module, deployed on a cloud server, processes the image data corresponding to the uncertain regions of the image through the feature enhancement extraction unit to generate deep feature data, and uses the deep feature data and primary defect detection data to incrementally learn the benchmark defect detection model through the model training unit, outputting the optimized defect detection model and model optimization strategy data. The high-precision detection execution module, deployed on the updated edge computing device, processes the pre-processed visual image data through the updated primary defect detection model to generate the final defect detection result data. The report and control unit then generates a defect report and control commands based on this result data.

Images