A lock paint fine flaw detection method and system of multi-directional visual edge computing

Abstract

The application discloses a lock paint fine flaw detection method and system for multi-directional visual edge computing. The system comprises a stereoscopic double-layer partitioned shell, an STM32 control board, an edge computing platform, an electric tray device, a five-way surround industrial camera and a coaxial illumination light source. The method comprises: collecting five-directional images and performing ROI division; calling an improved YOLO26-WTConv classification model to identify the lock type, wherein WTConv expands the receptive field through wavelet transform; calling an improved YOLO26-SPD-Conv detection model to locate the flaw, wherein SPD-Conv realizes lossless down-sampling through space-to-depth transformation; training the model by using a slice and whole image incremental learning strategy; and deploying the model on an edge platform after TensorRT acceleration for real-time inference. The application solves the problems of unclear imaging of small flaws of high-reflective curved surface locks, weak model fine-grained feature extraction and poor real-time performance of the edge, and the detection accuracy reaches more than 98%, meeting the real-time quality inspection requirements of industrial production lines.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection and edge computing technology, and in particular to a method and system for detecting minor defects in lock paint using multi-directional visual edge computing. Background Technology

[0002] In the lock manufacturing industry, the quality of paint spraying directly affects the corrosion resistance and appearance of the product. Common minor defects on the painted surface of locks include bubbles, particles, scratches, and pinholes. These defects are characterized by small target size, low contrast, random distribution, and severe interference from the high reflectivity of complex curved surfaces. Traditional manual visual inspection suffers from problems such as high subjectivity, low efficiency, fatigue, and high rate of missed detection, making it difficult to meet the high-precision and real-time inspection requirements of modern production lines.

[0003] In recent years, automated inspection technologies based on machine vision and deep learning have been gradually applied to the field of industrial surface defect detection. However, existing technical solutions still have the following shortcomings:

[0004] Limited image quality: The surface of the lock has complex curves and metallic luster. Traditional lighting solutions are prone to producing specular reflections and shadow blind spots, which can cause minor flaws to be masked and result in insufficient image contrast.

[0005] Limited inspection perspective: Single-direction or limited-angle cameras cannot cover the multi-dimensional surface of locks, resulting in missed detection of defects at edges and in hidden locations.

[0006] Weak ability to detect small objects: General object detection algorithms (such as the YOLO series) are not accurate enough in detecting small defects because they are prone to losing fine-grained features due to multiple downsampling operations.

[0007] Poor real-time performance at the edge: When deep learning models are deployed directly on embedded edge devices, the inference latency is high and the computing power consumption is large, making it difficult to meet the real-time requirements of industrial sites.

[0008] Lack of systematic solutions: Existing technologies lack a fully automated detection system that covers the entire process from underlying hardware control, multi-directional image acquisition, model improvement and training to accelerated deployment at the edge.

[0009] Therefore, there is an urgent need for a method and system for detecting minor defects in lock paint that can solve the above problems. Summary of the Invention

[0010] In view of this, the purpose of the present invention is to provide a method and system for detecting minor defects in lock paint using multi-directional visual edge calculation, so as to solve the technical problems in the prior art such as unclear imaging of minor defects in highly reflective curved workpieces, incomplete multi-directional coverage, weak ability of the detection model to extract fine-grained features, and poor real-time performance at the edge.

[0011] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting minor defects in lock paint using multi-directional visual edge computing, based on a lock paint minor defect detection system, the lock paint minor defect detection system including an STM32 control board, an edge computing platform, an electric tray device, an industrial camera, a lighting source, and an external interface panel;

[0012] Includes the following steps:

[0013] Step S1: The STM32 control board receives instructions from the edge computing platform, controls the electric pallet device to transport the lock to the inspection station, and triggers the five industrial cameras and coaxial lighting source to collect raw images of the lock from five directions.

[0014] Step S2: The edge computing platform divides the acquired image into ROIs, extracts the main lock area, and performs data augmentation processing;

[0015] Step S3: Call the preset lock classification model to perform inference on the main view image and output the lock model; the lock classification model is a network based on the improved YOLO26-WTConv, in which the WTConv module uses discrete wavelet transform to perform multi-level frequency decomposition and reconstruction of the feature map to expand the receptive field;

[0016] Step S4: Call the preset defect detection model to perform batch inference on the five images, and output the category, location coordinates and confidence of defects in each image; the defect detection model is a network based on the improved YOLO26-SPD-Conv, in which the SPD-Conv module achieves lossless downsampling through spatial-to-depth transformation to preserve the features of minor defects;

[0017] Step S5: Based on the reasoning results of steps S3 and S4, determine whether the lock is qualified and display the test results on the external interface panel in real time.

[0018] In a preferred embodiment, the ROI partitioning method in step S2 is specifically as follows:

[0019] S21: Obtain the spatial activation map: During the forward inference process of the lock classification model on the main view image, extract the feature response map output by the last convolutional layer in the classification branch. The spatial dimension of this response map is 1 / 32 of the original image.

[0020] S22: Generate activation mask: Convert the response map into a probability distribution using the Softmax function, select the activation map corresponding to the channel with the highest probability value as the saliency map, and perform bilinear upsampling of the saliency map to the original image size;

[0021] S23: Locating connected regions: Adaptive threshold segmentation is applied to the upsampled saliency map to obtain a binary mask, and then the connected region with the largest area in the mask is extracted as the candidate region of the lock body;

[0022] S24: Region Expansion: Based on the minimum bounding rectangle of the candidate region, expand the width and height of the original rectangle by 10% in each of the four directions of upward, downward, left and right to obtain the final ROI bounding box;

[0023] S25: Cropping Output: Crops the main lock area from the original image based on the ROI bounding box, and scales the cropped image to the standard input size of the defect detection model.

[0024] In a preferred embodiment, the lock classification model in step S3 is trained using the following method:

[0025] S31: Constructing the basic network: Using YOLO26-cls as the baseline classification network, the YOLO26-cls network includes a backbone network, a classification head, and a full pipeline aggregation and distribution structure;

[0026] S32: Embedding WTConv modules: Within the C3k2 Bottleneck structure of the high-level feature extraction layer of the backbone network, some standard 3×3 depth convolutional layers are replaced with WTConv modules;

[0027] S33: Forward computation of the WTConv module: for input features Figure X Two-dimensional Haar wavelet transform is applied to decompose the component into four frequency bands: low-frequency approximation component, horizontal high-frequency component, vertical high-frequency component, and diagonal high-frequency component. The wavelet decomposition is recursively performed on the low-frequency component, resulting in 2 to 3 levels of decomposition. After each level of decomposition, the spatial resolution is halved, while the receptive field is exponentially expanded. At each decomposition level, a 3×3 deep convolutional layer with a stride of 1 is applied to each of the four frequency bands without changing the size of each component. Inverse discrete wavelet transform is used to reconstruct the feature map level by level, starting from the deepest level, and the reconstruction results are weighted and aggregated with the feature map of the previous level.

[0028] S34: Training configuration: cross-entropy loss function is used, AdamW optimizer is used, and cosine annealing decay strategy is adopted. The training rounds are 600, the input image size is kept at high resolution, and supervised training is performed with lock model as the unique label.

[0029] S35: Model Output: After training, the model can output the predicted probability distribution of the lock model, which is used for real-time classification reasoning in step S3.

[0030] In a preferred embodiment, the defect detection model in step S4 is trained using the following method:

[0031] S41: Constructing the basic network: The YOLO26-P2 network is used as the benchmark detection network. This network contains four high-resolution feature detection heads with 4x downsampling, 8x downsampling, 16x downsampling and 32x downsampling, and has a feature pyramid network and path aggregation network structure.

[0032] S42: Replace downsampling layers: Replace all traditional convolutional layers and max pooling layers with stride 2 in the backbone network and feature fusion stage with SPD-Conv modules.

[0033] S43: Forward computation of the SPD-Conv module: Assume the input feature map size is H×W×C;

[0034] From spatial to deep layers: The input feature map is divided into four sub-feature maps based on the parity of the pixel coordinates, with each sub-map having a size of [size missing]. Then, the layers are concatenated along the channel dimension to obtain an intermediate feature map of size [size missing]. ;

[0035] Non-strut convolutional layer: Apply a standard convolutional layer with a kernel size of 3×3 and a stride of 1 to the intermediate feature map output by the SPD layer to reduce the number of channels from 4C to the preset number of output channels. This convolutional layer does not further reduce the spatial resolution.

[0036] S44: Construct a composite loss function: The bounding box regression loss uses CIoU loss, the classification loss uses binary cross-entropy loss, and the confidence loss also uses binary cross-entropy loss. The total loss is the weighted sum of CIoU loss, binary cross-entropy loss, and binary cross-entropy loss.

[0037] S45: Training configuration: SGD optimizer is used, with an initial learning rate of 0.01, momentum of 0.937, and weight decay of 0.0005. Mosaic data augmentation and MixUp strategies are enabled during training, and the training epochs are 600.

[0038] S46: Model Output: After training, the model can output bounding box coordinates, category labels, and confidence scores for defects such as scratches, dents, paint particles, and stains from the input image.

[0039] In a preferred embodiment, the lock classification model and the defect detection model employ a slice-and-whole-image incremental learning strategy during the training phase:

[0040] Phase 1: Cut the high-resolution original image into 1024×1024 pixel sub-patterns according to a set overlap ratio, and use the sub-patterns to train the model;

[0041] Second stage: Load the pre-trained weights obtained from the first stage of training, and use the complete original large image to perform secondary fine-tuning of the model.

[0042] In a preferred embodiment, the inference process in steps S3 and S4 is accelerated using TensorRT: the trained PyTorch model is converted into the TensorRT engine format, layer fusion, automatic kernel tuning and FP16 half-precision quantization are performed, and then it is deployed on an edge computing platform.

[0043] The present invention also provides a lock paint minor defect detection system based on multi-directional visual edge calculation, namely the lock paint minor defect detection system described in claims 1-6, comprising:

[0044] A three-dimensional, double-layered shell, with the interior horizontally divided into an upper and lower area;

[0045] An embedded control platform component is installed in the upper region, including an STM32 control board, an edge computing platform, and a power module;

[0046] A core detection work area is located in the lower layer area. This area has a closed, light-shielding structure and includes:

[0047] An electric tray unit is fixed to the bottom of the lower area;

[0048] At least five sets of industrial cameras are arranged around the inspection station of the electric pallet device, including one top-view camera and at least four side-view cameras;

[0049] Multiple sets of coaxial illumination sources, each coaxial illumination source is installed at the front end of an industrial camera, and the optical axis of the light source is coaxial with the imaging axis of the camera.

[0050] In a preferred embodiment, the upper and lower regions of the three-dimensional double-layer partitioned shell are physically isolated by a metal partition. The metal partition is provided with wire-passing holes, and a shielding ring for electromagnetic interference is provided at the wire-passing holes. The STM32 control board and edge computing platform are suspended on the metal partition of the upper region by multiple metal fixed pillars, and an air convection gap is left between them and the metal partition.

[0051] An external interface panel is provided on the rear or side wall of the upper region. The external interface panel integrates a power input interface, a debugging serial port, a USB interface, and an HDMI display interface.

[0052] In a preferred embodiment: the electric tray device includes a stepper motor, a high-precision lead screw, and a carrying tray; the upper surface of the carrying tray is covered with matte frosted paper; the bottom of the carrying tray is provided with a nut seat that cooperates with the lead screw, and the sides of the carrying tray are provided with sliders that slide in cooperation with the bottom guide rail of the lower area.

[0053] In a preferred embodiment: the coaxial lighting source includes a white mist LED light panel, a diffused light paper layer, and a light source housing; the light source housing is fixedly nested in front of the top-view camera; the four side-view cameras are arranged in a 90° interval ring around the detection station of the electric tray, and the optical axes of the cameras all point to the center of the tray.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] By using a five-camera surround layout and a coaxial diffuse reflection light source, the problem of multi-directional uniform illumination without blind spots in highly reflective curved surface locks is solved, significantly improving the contrast of minor imperfections.

[0056] The WTConv module is introduced into the classification network, and wavelet transform is used to obtain a large receptive field without significantly increasing the number of parameters, thereby enhancing the ability to distinguish the global structure and texture of the lock.

[0057] By introducing the SPD-Conv module into the detection network to replace the traditional stride convolution, lossless downsampling is achieved, which fully preserves the pixel-level features of minor imperfections.

[0058] We propose a slice-and-whole-image incremental learning training strategy, which ensures the model's high sensitivity to minor local flaws while also ensuring accurate perception of the global structure.

[0059] Accelerated using TensorRT and deployed on the Jetson Orin NX edge platform, the inference time for a single image is less than 16ms, and the total time for five-channel batch processing is less than 80ms, meeting industrial real-time requirements. Attached Figure Description

[0060] Figure 1 This is a diagram of the YOLO26-cls network structure according to a preferred embodiment of the present invention;

[0061] Figure 2 This is a diagram of the improved YOLO26-WTConv network structure according to a preferred embodiment of the present invention;

[0062] Figure 3 This is a diagram of the YOLO26-P2 network structure according to a preferred embodiment of the present invention;

[0063] Figure 4 This is a network structure diagram based on the improved YOLO26-SPD-Conv according to a preferred embodiment of the present invention;

[0064] Figure 5 This is a display of the dataset content for a preferred embodiment of the present invention;

[0065] Figure 6 This is a data slicing effect diagram of a preferred embodiment of the present invention;

[0066] Figure 7 This is a schematic diagram of the training strategy process according to a preferred embodiment of the present invention;

[0067] Figure 8 This is an isometric schematic diagram of a preferred embodiment of the present invention;

[0068] Figure 9 This is a cross-sectional schematic diagram of a preferred embodiment of the present invention;

[0069] Figure 10 This is a top view schematic diagram of a preferred embodiment of the present invention;

[0070] Figure 11 This is a schematic diagram of the electric pallet device according to a preferred embodiment of the present invention;

[0071] Figure 12 This is a flowchart illustrating the visual inspection system of a preferred embodiment of the present invention.

[0072] Figure 13 This is a flowchart of the STM32 software development process according to a preferred embodiment of the present invention;

[0073] Figure 14 This is a flowchart illustrating the edge device software development process according to a preferred embodiment of the present invention.

[0074] Figure 15 This is a schematic diagram of a power module according to a preferred embodiment of the present invention;

[0075] Figure 16 This is a schematic diagram of a four-channel DAC (one of which) according to a preferred embodiment of the present invention. Detailed Implementation

[0076] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0077] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0078] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0079] A method and system for detecting minor defects in lock paint using multi-directional visual edge computing, referenced Figure 1-16 This mainly includes a method for detecting minor defects in lock paint using multi-directional visual edge computing, a system for detecting minor defects in lock paint, and the deployment of edge AI models and software functionality development for terminals. These will be described in detail below:

[0080] 1. A method for detecting minute defects in lock paint using multi-directional visual edge computing

[0081] 1.1 Lock Classification Method Based on Improved YOLO 26

[0082] 1.1.1 Introduction to the YOLO26-cls Algorithm

[0083] In lock defect detection, rapid and accurate identification of the lock model is a crucial prerequisite for subsequent precise defect analysis. This invention uses YOLO26-cls as the basic classification model, and the network structure diagram is shown below. Figure 1As shown, its architecture design deeply aligns with the industrial inspection's dual pursuit of "lightweight" and "high precision." YOLO26-cls, the latest evolution of the YOLO series launched in 2026, completely reconstructs the end-to-end inference process by introducing a fully pipelined aggregation and distribution mode. This model greatly simplifies the computation graph structure by eliminating redundant non-maximum suppression logic and distributed focal length loss commonly found in traditional classification networks at the underlying algorithm level. This significantly lowers the deployment threshold on edge computing devices or embedded hardware and substantially reduces inference latency. In the backbone network design, YOLO26-cls integrates the advanced MuSGD optimizer. This optimizer, with its unique multi-stage weight update mechanism, effectively enhances the model's stability when extracting complex texture features, especially demonstrating strong discriminative power when handling subtle differences in key lock parts, thus avoiding recognition errors caused by overfitting or gradient oscillations. Furthermore, the YOLO26-cls classification head employs a combination of global average pooling and lightweight fully connected layers. This design not only maintains high sensitivity to global image features, ensuring accurate model identification from various perspectives, but also frees up more computing resources by reducing the number of parameters. This comprehensive optimization from backbone to head enables the YOLO26-cls to maintain high classification accuracy while achieving extremely high frame rates. It fully meets the stringent performance requirements of real-time lock classification and dynamic detection on modern industrial production lines, providing solid technical support for building a fully automated and intelligent lock quality control system.

[0084] 1.1.2 Network Design Based on Improved YOLO26-WTConv

[0085] The core improvement of this invention for lock classification tasks lies in replacing some standard depthwise convolutions in the basic YOLO26-cls backbone network with improved wavelet convolutions—the WTConv module. Traditional convolutional neural networks typically rely on continuously stacking convolutional layers or increasing the kernel size to expand the receptive field, but this leads to an exponential increase in computation and is prone to model degradation. Lock images have complex textures and diverse geometric structures, and a limited local receptive field often makes it difficult to accurately capture the essential features of lock models in complex backgrounds. Therefore, this invention introduces WTConv, utilizing the frequency decomposition characteristics of discrete wavelet transform to achieve a significant expansion of the network's receptive field without adding extra parameter burden.

[0086] like Figure 2As shown, in a specific improved implementation, this invention embeds the WTConv module into the high-level feature extraction layer of the YOLO26-cls backbone network, particularly in the Bottleneck structure. The WTConv operation first uses multi-level wavelet decomposition to spatially split the input feature map into low-frequency approximation components and high-frequency detail components. This frequency decoupling mechanism allows the network to "differentiate" features at different scales: the low-frequency components carry the overall outline and macroscopic topology of the lock, while the high-frequency components precisely pinpoint the distribution of minor flaws on the lock's edges and surface. Compared to standard convolution performing a sliding window operation on a single scale, WTConv performs convolution operations on different frequency sub-bands in parallel, and then uses inverse discrete wavelet transform for feature reconstruction. This process is physically equivalent to feature sampling over a very large spatial range, thus providing the model with an extremely large effective receptive field, enabling it to examine lock categories from a global perspective.

[0087] The reason for choosing WTConv to improve YOLO26 is based on the specific requirements of lock classification tasks, which are sensitive to both "global structure" and "local texture." Traditional deep networks often lose crucial high-frequency details as the downsampling factor increases, leading to misclassification of similar lock models. WTConv effectively compensates for this deficiency by capturing multi-scale information in the wavelet domain. Furthermore, due to the energy-concentrating nature of wavelet transform, the improved network can focus more on meaningful feature regions in the image, suppressing the influence of background noise on the classification results. This design significantly improves the model's spatial modeling ability for large-scale targets while maintaining inference speed comparable to the native YOLO26 architecture.

[0088] Experimental analysis shows that the improved YOLO26-WTConv exhibits stronger robustness in lock classification tasks. Visual heatmap analysis reveals that the improved model's activation region more comprehensively covers the overall shape of the lock, rather than being limited to local high-contrast edges. This increased receptive field directly translates into improved classification accuracy, especially in handling lock deformation caused by camera angle shifts or in environments with uneven lighting, where the improved model demonstrates extremely high feature aggregation efficiency. The introduction of WTConv not only enhances the model's discriminative power against complex topological features...

[0089] 1.1.3 ROI Partitioning Method

[0090] After initial identification of lock types, accurately locating the detection area is crucial for improving subsequent defect detection accuracy. This invention proposes a Region of Interest (ROI) partitioning method based on classification region feedback, utilizing the spatial feature responses of an improved classification model. Because the improved YOLO26-WTConv has a larger receptive field, its high-level feature maps contain extremely rich spatial location information. By analyzing the spatial activation maps in the classification branches, the algorithm can automatically locate the geographic coordinates of the lock body in the image. Specifically, the system extracts the most significant connected regions during the classification prediction process and expands them using a preset scaling factor, thereby generating a candidate bounding box that completely covers the lock body. This ROI region is then cropped and input into a subsequent detection model specifically designed for minor defects. This two-stage cascaded strategy not only effectively filters out a large amount of background interference information in the image but also significantly reduces the search space of the defect detection model, allowing subsequent models to focus on high-resolution local detail features, thus significantly improving the recall and accuracy of lock surface defect detection.

[0091] 1.2 A Minor Defect Identification Method Based on Improved YOLO26

[0092] 1.2.1 Introduction to the YOLO26-P2 Algorithm

[0093] In lock surface defect detection tasks, defects such as tiny scratches, rust spots, or holes typically occupy only a few pixels in the image and are often hidden in the complex textured background of the ROI area. For example... Figure 3 As shown, to accurately capture these extremely small features, this part of the research is based on the YOLO26-P2 algorithm equipped with a P2 high-resolution feature detection head. Traditional detection models usually tend to make final predictions on the P3, P4, and P5 feature layers, which correspond to feature maps downsampled by 8, 16, and 32 times, respectively. This inevitably leads to the severe compression or even complete disappearance of the fragile spatial location information and subtle geometric features of tiny defects during multiple consecutive downsampling processes, resulting in the model exhibiting obvious missed detections when facing pixel-level extremely small defects.

[0094] To effectively address this issue, the YOLO26-P2 algorithm introduces a P2 feature layer, i.e., a feature map with 4x downsampling. This successfully preserves a higher spatial resolution feature map in the backbone network, enabling the model to directly acquire richer shallow geometric features and precise spatial location information. This allows for the detection of extremely subtle texture changes and edge anomalies during the feature extraction stage. Furthermore, to address the increased computational cost associated with introducing the P2 layer, this invention features a lightweight modification to the detection head. It utilizes a cross-scale feature fusion mechanism, combining FPN and PAN structures to efficiently couple the abstract semantic information of the deep network with the edge detail information of the shallow network, effectively controlling the model's inference overhead while maintaining detection accuracy. This optimized architecture not only enhances feature representation capabilities but also ensures that the model can accurately identify pixel-level subtle defects when processing local images cropped by the ROI segmentation method in the previous section, laying a solid algorithmic foundation for subsequent precise detection of lock quality.

[0095] 1.2.2 Network Design Based on Improved YOLO26-SPD-Conv

[0096] In industrial lock surface defect detection, the completeness of feature extraction, the ability to represent multi-scale features, and the efficiency of algorithm operation are the core factors determining the final detection accuracy and industrial feasibility of deep learning models. Regarding the YOLO26-P2 network architecture introduced in 1.2.1, although it establishes a direct mapping channel for shallow features by introducing the P2 high-resolution detection head, thus preserving valuable shallow spatial geometric information to some extent and significantly enhancing the network's initial perception and localization capabilities for extremely small targets, a deeper analysis of its forward propagation mechanism reveals that the network still uses traditional stride convolution and pooling strategies when performing backbone feature downsampling and spatial dimension compression. This "jump-style" feature sampling method, while expanding the receptive field and extracting higher-order semantics, is accompanied by crude information truncation and the fragmentation of local contextual relationships, failing to achieve pixel-level feature preservation. For minute defects on the lock surface, such as extremely low contrast, tiny scratches, pinhole-like rust spots, and highly concealed microcracks, which occupy only a few pixels in geometric size, this discontinuous sampling process inevitably leads to the excessive smoothing or even complete erasure of their weak edge textures and topological structures. As the network depth increases, these defect feature flows, already extremely fragile in shallow layers, are easily submerged by background noise during feature map propagation after multiple rounds of indiscriminate spatial compression, resulting in severe "feature dilution" and information attenuation. This significantly reduces the deep network's ability to abstractly represent minute defects, creating a serious risk of missed detections for subsequent accurate identification and bounding box regression. To further improve the ability of YOLO26-P2 to represent fine-grained defects, this invention introduces the SPD-Conv module into the YOLO26-P2 architecture to reconstruct the downsampling structure in the network, constructing an improved YOLO26-SPD-Conv network, as follows... Figure 4 As shown.

[0097] The core idea of ​​SPD-Conv is to replace traditional stride convolution with a space-to-depth (SPD) transformation to achieve feature downsampling with "no information loss". Specifically, when the network needs to perform downsampling by a factor of S, the SPD module first spatially rearranges the input feature map. For example, the SPD layer divides the original feature map into four complementary sub-feature maps based on the parity of pixel coordinates. Each sub-map fully preserves the local spatial information of the original input. These sub-feature maps are then concatenated along the channel dimension, achieving a mapping transformation from spatial to channel dimensions. In this process, although the feature map size is reduced, the original pixel information is not discarded, thus maximizing the preservation of edge details, texture structure, and grayscale variation features of lock defects. Compared to traditional stride convolution, which only sparsely samples local pixels, the SPD operation ensures that every pixel in the ROI region participates in subsequent feature learning, fundamentally preventing minute defects from being "evaporated" during the downsampling stage.

[0098] After completing the SPD feature rearrangement, to alleviate the computational pressure caused by the rapid increase in the number of channels and to further achieve feature fusion, this invention introduces a standard convolutional layer with a stride of 1 after the SPD layer. This convolutional layer no longer undertakes the spatial compression task, but instead serves as a feature integration module, performing nonlinear fusion and dimensionality reduction mapping on the high-dimensional channel information output by the SPD. Since the features after SPD transformation have strong spatial correlations, a convolution with a stride of 1 can effectively establish semantic connections between different channels without sacrificing spatial resolution, thereby obtaining a more stable and high-quality feature representation. In the improved YOLO26-SPD-Conv network, this structure is widely used in the YOLO26-P2 backbone network and key downsampling nodes in the feature fusion stage, replacing the original Stride=2 convolutional layer. At the same time, the original P2 detector head structure of YOLO26-P2 is retained, enabling the model to maintain high-resolution feature output while further enhancing its fine-grained perception of minute defects.

[0099] From a network structure perspective, YOLO26-SPD-Conv inherits the multi-scale detection framework of YOLO26-P2 and combines it with the information preservation advantages of SPD-Conv to achieve synergistic optimization of "high-resolution detection" and "lossless downsampling". The P2 detector head is responsible for preserving shallow spatial details, while SPD-Conv is responsible for preventing the loss of key information during downsampling. Together, they improve the model's responsiveness to small targets. Especially in industrial scenarios where lock surfaces have complex reflections, high-frequency textures, and weak contrast defects, SPD-Conv can effectively enhance the fidelity of underlying texture features, enabling deep networks to still perceive the spatial distribution information of subtle defects.

[0100] Experimental results show that the improved YOLO26-SPD-Conv network, based on YOLO26-P2, exhibits stronger robustness and detection accuracy in detecting minor defects in locks. Feature response heatmap analysis reveals that the improved model not only focuses more completely on the defective areas of the lock but also maintains stable feature activation responses even in scenarios with minor scratches, minor rust spots, and low-contrast defects. Compared to the original YOLO26-P2 network, SPD-Conv significantly reduces the false negative rate for small targets and enhances the model's generalization ability in complex industrial environments. This improved strategy, based on "complete preservation of spatial information," provides a high-fidelity feature foundation for subsequent detection heads to achieve more accurate bounding box regression and class discrimination, thereby enabling more precise and reliable intelligent detection of lock surface quality.

[0101] 1.3 Loss Function Construction

[0102] 1.3.1 Lock Classification Loss Function

[0103] In the first stage of the lock model recognition task, the improved YOLO26-WTConv model essentially performs a multi-class classification task. To enable the model to accurately learn the subtle feature differences between different lock models, this invention uses the cross-entropy loss function as the guiding function for the classification branch. The cross-entropy loss measures the difference between the probability distribution predicted by the model and the true label distribution, and its formula is as follows:

[0104]

[0105] In the formula, This indicates the total number of lock model categories. One-hot encoding of the real label. This represents the predicted probability after the model's output layer has been processed by the Softmax function. By minimizing this loss term, the model can continuously optimize the wavelet domain convolution parameters in the WTConv module, thereby improving its ability to discriminate between the macroscopic structure and local texture of locks in complex contexts.

[0106] 1.3.2 Loss Function for Minor Defect Detection

[0107] In the second stage, which involves detecting minor defects within the ROI region, the improved YOLO26-SPD-Conv model needs to simultaneously handle defect localization and classification. Therefore, its overall loss function is a weighted combination of bounding box regression loss, classification loss, and confidence loss, as shown in the following formula:

[0108]

[0109] In the formula, , , These represent the weighting coefficients for each loss term. Regarding the bounding box regression loss, considering that lock defects are extremely sensitive to overlap, this invention adopts the CIoU loss function. CIoU further considers the distance between the center points of the predicted box and the ground truth box, as well as the consistency of the aspect ratio, based on IoU. This allows the improved model to converge more stably to extremely small defect targets after introducing the P2 high-resolution detector head.

[0110] Classification loss regarding defect categories With object confidence loss This invention employs the binary cross-entropy loss function. For defect detection, the BCE loss effectively handles multi-label classification problems, i.e., determining whether a specific type of defect exists within the ROI region. Its formula is as follows:

[0111]

[0112] In the formula, y represents the actual presence of the defect. This represents the predicted score after Sigmoid activation. Combined with the SPD-Conv module's ability to preserve pixel-level information, this loss function guides the model to focus on low-contrast features that are easily lost during downsampling, thus significantly improving the detection accuracy of minor scratches, holes, and other defects. Through the constraints of this composite loss function, the system can achieve precise localization and robust identification of lock defects under high-resolution feature maps.

[0113] 1.4 Dataset Construction and Training Strategies

[0114] 1.4.1 Data Acquisition and Labeling

[0115] High-quality datasets are the core foundation for training deep learning models, and their quality directly determines the upper limit of the model's ability to discriminate complex defects. This invention utilizes an industrial camera to collect samples within a lock inspection chamber. To ensure high-fidelity representation of features such as metal textures, micro-scratches, dotted rust, and cracks on the lock surface, a self-made fog-like light source was specially configured during the acquisition process. By precisely adjusting the incident angle and spatial distribution of the light, the severe specular reflection interference easily caused by the metal material was cleverly avoided, thereby obtaining original image materials with significant contrast and clear detail levels, such as... Figure 5As shown. In the data preprocessing stage, this invention uses the Labelme annotation tool to perform refined rectangular bounding box annotations on different lock models and their corresponding surface defects, constructing a JSON-formatted metadata pool containing spatial coordinates and category semantics. Subsequently, through an automated parsing script, these heterogeneous JSON annotation information are batch-mapped and converted into the TXT standard format adapted to the YOLO series framework. The dataset is strictly divided into training, validation, and test sets according to scientific statistical proportions to ensure the independence and fairness of the model evaluation process. Addressing the common problems of uneven data distribution and positive / negative sample imbalance in industrial settings, this invention introduces a complete data augmentation strategy. By integrating spatial rotation and flipping, image brightness and contrast adjustment, and random Gaussian noise addition, the training set is dynamically expanded in multiple dimensions. This "data increment" approach not only effectively mines the potential representation space of existing samples but also significantly improves the generalization ability and anti-interference robustness of the detection model in complex production environments such as lighting fluctuations and workpiece position deviations, laying a solid data foundation for subsequent high-precision defect identification.

[0116] 4.4.2 Image Slicing Preprocessing Strategy

[0117] Considering the high resolution of the original lock image used in this invention, directly scaling it to the model's standard input size would cause minor flaws to lose crucial pixel-level features during severe downsampling. To achieve efficient training without sacrificing flaw details, this invention proposes a preprocessing strategy based on image slicing. Specifically, the algorithm slices the original high-resolution image into a series of 1024×1024 pixel sub-patterns with a certain overlap ratio, as shown in the image. Figure 6 As shown, this slicing strategy ensures that each local sub-image maintains extremely high spatial resolution, enabling the aforementioned improved YOLO26-P2 detector head to capture more microscopic defect information. This approach not only alleviates the limitations of GPU memory configuration for training large images, but more importantly, it forces the model to perform deep feature mining on tiny targets from a "local perspective," fundamentally solving the problem of weak feature responses of small targets in deep neural networks.

[0118] 1.4.3 Incremental Learning Method for Whole Graphs

[0119] While slice training can significantly improve the detection sensitivity of small targets, the slicing operation severs the global spatial relationships of the image, which may lead to a bias in the model's understanding of the overall topology of the lock. Therefore, this invention further introduces a whole-image incremental learning training phase on top of slice training. Figure 7As shown, after the model initially acquires the ability to identify minor local flaws, it undergoes a second fine-tuning process using a full-size original image by loading pre-trained weights. In this stage, the model recalibrates the perception weights of the classification and localization branches regarding the overall outline of the lock using a full-size image as input, enabling the ROI segmentation module to more accurately anchor the target object. This two-stage learning strategy of "local refinement followed by global regression" ensures both extreme sensitivity to pixel-level minor flaws and a grasp of the lock's global features, thus achieving closed-loop accurate detection of the lock and its flaws even in complex backgrounds.

[0120] This section constructs a two-stage lock detection system based on an improved YOLO26. In the classification stage, the introduction of WTConv effectively expands the network's receptive field, enhancing its ability to capture global contextual information. In the recognition stage, SPD-Conv and the P2 detection head collaboratively optimize the representation of small targets, preserving the spatial geometric features of minute defects to the greatest extent possible. Furthermore, by combining slicing and incremental learning strategies, the computational bottleneck and the problem of missed defect detection under high-resolution images are specifically addressed. Experimental results demonstrate that this scheme exhibits excellent detection accuracy and robustness in practical industrial quality inspection scenarios, possessing extremely high practical value.

[0121] 2. Deployment of AI models and software function development at the edge of the terminal

[0122] This paper focuses on the design and implementation process of a lock surface visual inspection platform based on edge computing. The system utilizes an NVIDIA Jetson Orin NX 8GB edge computing development board, combined with a Python 3.10 development environment and a PyQt5 graphical interface framework, to construct a comprehensive inspection platform integrating device control, model inference, and result display. The core visual inspection algorithm adopts the lightweight and efficient YOLO series models to meet the dual requirements of inspection accuracy and real-time performance in industrial settings. This invention will be discussed in detail from four aspects: edge computing device environment setup, system functions and inference process, user interface design, and software function development, and a summary of the invention's content will follow.

[0123] 2.1 Setting up the edge computing device environment

[0124] In industrial vision inspection scenarios, edge computing platforms need to possess powerful parallel processing capabilities and highly scalable interfaces to meet the demands of multi-camera data acquisition, collaborative control of mechanical actuators, and real-time inference of deep learning models. Given the stringent requirements of timeliness and environmental adaptability for quality inspection tasks, the system selects the NVIDIA Jetson Orin NX as its core computing node. This platform is equipped with an NVIDIA Ampere architecture GPU, containing dedicated Tensor Cores, providing up to 70 TOPS of AI computing power. Its core advantage lies in its 8GB high-speed unified memory architecture, which eliminates the bandwidth bottleneck of traditional heterogeneous computing through physical memory sharing between the CPU and GPU, ensuring the system can efficiently run the YOLO detection model and the two-stage classification network, providing a solid hardware guarantee for real-time processing of multi-channel data. The system hardware platform follows a modular and highly collaborative design principle, mainly comprising four core components: edge computing devices, vision sensors, transmission structures, and a human-machine interaction module.

[0125] The vision sensors employ multiple high-definition industrial cameras, arranged in a ring around the fully enclosed inspection chamber. The transmission structure is an electric pallet system driven by a stepper motor, responsible for the automatic entry of products to be inspected into the chamber, precise positioning of the vision station, and smooth exit after inspection, effectively reducing mechanical vibration and positional deviation caused by manual loading and unloading.

[0126] The human-machine interaction module is implemented through an external high-resolution industrial touch screen, which can not only display the inspection screen, bounding box regression results and defect classification statistics in real time, but also allow operators to complete the online configuration and process control of core indicators such as camera exposure parameters, inspection confidence threshold, and motor running speed through the touch interface.

[0127] In terms of software environment configuration, the system is built on the Ubuntu operating system and deploys a Python 3.10 runtime environment, laying the foundation for modular writing of core algorithms and compatible calling of third-party libraries. Addressing the challenge of accelerating inference for deep learning models on the edge, the system integrates NVIDIA's official visual computing acceleration technology stack: by installing the CUDA parallel computing architecture and cuDNN neural network acceleration library, hardware-level acceleration of underlying tensor operations is achieved; furthermore, the TensorRT inference optimization engine is introduced, which transforms YOLO series models into highly optimized local engine files through layer fusion, automatic kernel adjustment, and FP16 half-precision quantization, significantly reducing inference latency and GPU memory usage without sacrificing detection accuracy.

[0128] For user interface development, the system installed the PyQt5 library and adopted a multi-threaded programming architecture to completely separate the energy-intensive image acquisition and TensorRT inference threads from the main UI refresh thread. This fundamentally eliminated lag and deadlock issues in the interactive interface, achieving a modern and responsive human-computer interaction. Through the deep integration and collaborative optimization of the above hardware and software environments, a stable, efficient, and low-latency edge computing vision inspection platform was successfully built.

[0129] 2.2 Function and Reasoning Flow

[0130] This invention details the overall functional logic and core reasoning process of an industrial product visual inspection system. The system process design balances ease of operation with rigorous inspection, ensuring efficient and accurate execution of every step from product loading to inspection report generation.

[0131] The overall design process of the device is as follows: Figure 12 As shown. First, after starting the equipment, the system enters the initialization phase, waiting for various sensors, actuators, and underlying software services to load. After initialization is complete, the operator needs to enter account information on the interface for login verification. Logging into the system is used not only for access control but also for tracing the responsible parties for testing different batches of products. If login fails, the system will prompt for re-entry; if login is successful, the system will proceed to the product parameter configuration stage.

[0132] During the parameter configuration phase, the operator needs to select the color and material of the product to be inspected. Based on the selected color and material, the system performs two key operations in parallel: first, loading a PT model matching the product; and second, configuring the optimal lighting conditions and camera exposure time to ensure the quality of subsequent image acquisition. After configuration, the system controls the extension of the electric tray, and the operator places the product to be inspected within the tray's effective area and triggers the operating button. Subsequently, the tray carrying the product automatically enters the inspection chamber.

[0133] Once the system detects that the tray has reached the designated detection position, the vision acquisition module is activated, controlling multiple cameras to poll and acquire high-definition photos of the product from five directions. After image acquisition is complete, the system enters a dual-stream parallel inference stage. On one hand, the system extracts the first main view image and inputs it into the image classification model for rapid product type determination and verification to prevent mixed materials or incorrect model numbers. On the other hand, the system acquires all five multi-angle images and inputs them in batches into a target detection and defect recognition model based on the YOLO architecture. This model utilizes its powerful feature extraction network to accurately locate and classify various defects on the product surface, such as scratches, pits, and stains.

[0134] The system waits for both the CLS and DET models to complete their inference tasks before making a comprehensive judgment on the two results. Finally, the system outputs the detection results to the UI interface in real time, synchronously updating and saving the product inspection report. After the inspection process is closed, the electric tray automatically exits, the operator removes the product, and a single visual inspection cycle is completed.

[0135] 2.3 Software Function Development

[0136] 2.3.1 Embedded Software Development for the Core Control Board

[0137] The underlying control board is based on the STM32 microcontroller, and the software is developed using the Keil uVision5 IDE and the HAL firmware library configured with STM32CubeMX. The core tasks of the underlying embedded program are to implement high-precision timing control, closed-loop regulation algorithms, and highly reliable industrial communication. For example... Figure 13 As shown, its architecture design mainly includes the following three core modules:

[0138] Multi-channel stepper motor timing control module: To meet the stringent requirements of precise positioning of the electric tray driven by a 57-stepper motor, the software is configured with an STM32 hardware timer to generate high-frequency, high-precision PWM pulse sequences. By enabling the timer's DMA mode, the pulse output does not occupy CPU interrupt resources, thus ensuring the continuity and smoothness of the pulse sequence during frequent tray starts, stops, accelerations, decelerations, and speed adjustments, effectively suppressing instantaneous mechanical shocks.

[0139] Light Source Brightness PID Closed-Loop Control Module: For the digital drive of coaxial diffuse reflection light sources, the software constructs a digital PID closed-loop adjustment algorithm based on an on-chip DAC and an external ADC. In the main loop, the system samples the output voltage of the feedback pin at regular intervals using the ADC, compares it with the set target value, calculates the correction amount using an incremental PID algorithm, and dynamically adjusts the output level of the DAC pin. This achieves rapid injection compensation for the XL4015E1 feedback loop, ensuring constant brightness of the light source under complex operating conditions.

[0140] Two-way communication and state machine interaction module: The serial communication adopts a design architecture of "UART interrupt reception + internal circular buffer". The software internally designs a high-efficiency instruction parsing state machine. When a motion or dimming instruction is received from the Jetson Orin NX platform, the state machine verifies the validity of the data frame header, frame trailer, and checksum. Upon successful verification, it immediately triggers a hardware action and, after completion, packages and sends back the state data, providing a reliable underlying hardware synchronization signal for upper-layer business logic.

[0141] 2.3.2 Edge Computing Platform Software Function Development

[0142] This invention explores the specific development and implementation of software functions based on the Jetson Orin NX platform. The software system is developed using Python 3.10 under the Ubuntu operating system, and the overall architecture adopts a modular and multi-threaded design to fully leverage the concurrent processing advantages of multi-core CPUs and GPUs.

[0143] like Figure 14 As shown, the code execution architecture is mainly divided into four core threads: hardware control and sensor communication thread, image acquisition and preprocessing thread, AI model inference thread, and PyQt5 GUI main thread.

[0144] (1) Hardware Control and Communication Module: The hardware control module mainly interacts with the underlying hardware such as the electric tray and the light source controller in collaboration with the STM32. The system sends control commands through the serial port and receives return signals of the execution results from the lower-level machine. To ensure the real-time response of the system, hardware communication is encapsulated in an independent background thread and synchronizes data with the main program through a thread-safe message queue.

[0145] (2) Image Acquisition and Preprocessing Module: This module manages multiple cameras. The system uses the Miicam library to call the camera interface, triggering simultaneous or polling capture of multiple cameras after the tray reaches the designated position. Before entering the AI ​​model, the acquired raw high-resolution images need to undergo preprocessing operations, including image normalization and color space conversion, to meet the input tensor format requirements of the YOLO model and classification model. The preprocessing process utilizes the hardware acceleration engine of the Jetson platform, significantly reducing the CPU load.

[0146] (3) AI Model Inference Module: Inference is the core of the system's computing power, involving the CLS classification model and the DET defect detection model. During development, in pursuit of the ultimate inference speed, the system did not directly use the native PyTorch model, but instead optimized the computational graph and performed INT8 / FP16 quantization conversion on the trained YOLO series models through TensorRT. In the inference thread, the system schedules models according to the process logic. First, the lightweight CLS model is called to perform classification inference on the first image; at the same time, five images are packaged into a batch of data and sent to the DET model accelerated by TensorRT. The YOLO model efficiently extracts features on the GPU and outputs tensors containing defect categories, coordinates, and confidence scores. The post-processing module performs non-maximum suppression parsing on these tensors and finally encapsulates the exact defect information into structured data and returns it to the main thread.

[0147] (4) GUI and Business Logic Module: The GUI module is developed based on PyQt5 and runs in the main thread, responsible for rendering the interface and responding to user interaction events. To prevent the interface from lagging during model inference or image processing, the system adopts PyQt's QThread and signal-slot mechanism. After completing their respective tasks, the background inference thread and hardware control thread transmit data such as state updates, image frames, and detection results to the slot function of the main thread by emitting signals. The slot function is responsible for refreshing the UI components. This design ensures the smoothness of the interface and the stability of the system.

[0148] This section details the design and implementation process of an industrial vision inspection edge computing platform. First, it introduces the hardware environment setup based on Jetson Orin NX and the configuration of the Python 3.10 software runtime environment. Second, considering actual industrial needs, it outlines the complete workflow from device startup, login, parameter configuration, image acquisition to AI dual-stream parallel inference. Subsequently, it introduces the intuitive and efficient graphical user interface design developed based on PyQt5. Finally, it delves into the modular development and implementation of the system software functions, focusing on the application of multi-threaded architecture, hardware communication, TensorRT model accelerated inference, and the PyQt signal-slot mechanism. Through the design and development of this invention, a high-precision, low-latency, and easy-to-operate edge computing vision inspection system has been successfully constructed.

[0149] 3. Terminal Embedded Control Board Circuit Design

[0150] 3.1 Terminal Embedded Control Board Circuit Design

[0151] The underlying control board of this part is based on an STM32 microcontroller and integrates precision power conversion, four-channel adjustable constant voltage light source driver, stepper motor control interface and multiple industrial communication circuits.

[0152] 3.1.1 Industrial-grade multi-channel power supply module design

[0153] Industrial sites typically provide 24V DC power. To meet the varying voltage requirements of different modules within the system, such as… Figure 15As shown, the control board employs a multi-stage high-efficiency step-down system. The system power input is equipped with a filter network consisting of electrolytic capacitors and inductors to smooth voltage fluctuations and filter out high-frequency switching noise. The main power module then steps down the 24V to 12V to power the subsequent drive circuits. For the core microcontroller and sensitive analog circuits, the system further converts the 12V to 3.3V and 5V using low-dropout linear regulators. Near each power pin of the microcontroller, a combination of ceramic and tantalum capacitors is arranged. This layout utilizes the impedance frequency characteristics of different capacitors to effectively suppress high-frequency interference caused by digital logic switching, ensuring power integrity and reference voltage stability in complex electromagnetic environments.

[0154] 3.1.2 Design of Digital Light Source Drive and Transmission Interface

[0155] The light source and transmission structure are the core execution units of this detection system, and their circuit design directly determines the quality of image acquisition and the accuracy of mechanical positioning. Regarding light source control, this design abandons the PWM dimming scheme, which is prone to high-frequency flicker, and instead adopts analog voltage control technology based on four-channel DAC feedback injection. For example... Figure 16 As shown, the schematic integrates four independent XL4015E1 constant voltage drive circuits. The output signal from the STM32's DAC pin is injected into the feedback pin of the power supply chip through an RC coupling network, thereby dynamically intervening in the balance point of the voltage regulation loop and forcing the output voltage to change linearly and smoothly within the range of 6V to 12V. To achieve more precise light field control, the output voltage is introduced to the ADC sampling pin through a voltage divider resistor, constructing a low-level software PID closed-loop algorithm to ensure the consistency of light source brightness under fluctuating load conditions. At the mechanical transmission level, the control board brings out standard control interfaces such as PWM, DIR, and EN. It uses the STM32's hardware timer to generate high-frequency, high-precision pulse sequences to drive a 57-stepper motor to complete the precise rotation of the lock tray, thus meeting the stringent spatial positioning requirements of multi-directional visual inspection.

[0156] 3.1.3 System Interaction and Communication Circuit Design

[0157] The communication circuit, acting as the link between the control board and external systems, plays a crucial role in distributing detection commands and transmitting hardware status feedback. The system employs a redundant communication architecture. The debugging branch utilizes a CH340G-USB to serial chip, employing charge pump logic to convert TTL levels to the USB standard, supporting real-time logic monitoring and firmware burning via a PC. In the underlying interaction link, the STM32 establishes a data connection with the NVIDIA Jetson edge computing platform through a high-speed UART interface. Considering the potential for electrostatic discharge (ESD) or transient surges in industrial environments, ESD protection is specifically added to the communication interface, effectively limiting signal levels within a safe voltage range. This communication design, combining signal integrity optimization with physical protection, ensures highly reliable transmission of detection commands in continuous operating environments.

[0158] As the hardware foundation for the entire system, this control board undertakes the core communication and control tasks between the edge computing platform and the actuators. On one hand, it receives motion control commands from the edge computing platform and converts them into control signals that can directly drive field devices. On the other hand, it also needs to collect and provide feedback on the key hardware status during system operation in real time, providing stable and accurate data support for the upper-level control logic. Through the rational division of the board's circuit functions and the overall optimization of the PCB layout, not only is the reliability and anti-interference capability of control signal transmission improved, but a solid foundation is also laid for the stable operation and functional expansion of the subsequent system.

[0159] 3.2 Design of Light Source for Image Acquisition on Painted Surface

[0160] In visual inspection systems, the quality of the light field directly determines the feature representation capability of the original image. This is especially true for the painted surfaces of locks with unique geometric shapes and metallic luster, where the design of the lighting scheme is crucial. Locks, due to their complex curved surfaces, edges, and keyhole structures, are prone to creating shadow blind spots under illumination. Simultaneously, the high reflectivity of the paint layer often leads to specular reflection, forming locally overexposed "flares" in the image, obscuring fine particles, bubbles, or scratches. To address this challenge, this invention independently designed and implemented a dedicated coaxial visual lighting system. The core of this design lies in aligning the emission axis of the light source with the imaging axis of the camera, allowing the light to be projected perpendicularly and uniformly onto the workpiece surface, thereby minimizing the problems of uneven shadows and reflections caused by angular tilt.

[0161] To further optimize imaging and mitigate direct light interference from point light sources, white, frosted LED beads were selected as the light-emitting element within the light source. Compared to traditional high-brightness LEDs, frosted LED beads offer better initial diffusion, reducing beam directivity. At the front end of the light path, a diffuser paper was used as a diffuser cover, utilizing the unevenly distributed fiber structure within the paper to perform multiple diffuse reflections of the light. Through this multi-diffusion mechanism, the original point-like intense light is transformed into a uniform surface light source, effectively reducing the direct intensity of light on the metallic paint surface. This ensures soft and evenly distributed light in the image, significantly improving the feature contrast of subtle imperfections against complex backgrounds.

[0162] Furthermore, to ensure the mechanical stability and optical axis coupling accuracy of the optical system, this invention simultaneously designed and manufactured a dedicated light source housing. This housing structure, precisely calculated, can be stably installed and nested in front of the industrial camera lens, achieving precise alignment between the light source and the camera's optical axis through physical positioning. This integrated coaxial housing design not only protects the internal optical components but also reduces interference from stray light from the external environment through its compact structure, ensuring high clarity and consistency in the acquired images. This light source design successfully solves the detection challenges posed by the high reflectivity and complex shape of lock surfaces, providing a high-quality image source for accurate defect identification by subsequent edge computing platforms.

[0163] 3.3 Electric Tray Design for Locks

[0164] To achieve smooth multi-angle switching and precise positioning of locks during the inspection process, this system independently designed and developed a high-precision electric tray device. This device, as the core of the mechanical execution, is responsible for driving the locks under inspection along a predetermined trajectory within the visual acquisition area. The system uses a 57 series stepper motor as its power source, which possesses high output torque and stepping accuracy, meeting the dynamic response requirements of workpieces with a certain mass, such as locks, during the start-up and stop phases. To ensure smooth motor operation and suppress low- and mid-frequency vibrations, the system employs a DM542 digital stepper driver. Based on advanced average current control and multi-microstepping technology, this driver can precisely subdivide the motor's step angle, effectively reducing the instantaneous mechanical impact caused by stepping pulses, thereby ensuring the physical stability of the platform during image acquisition.

[0165] At the level of mechanical transmission, such as Figure 11As shown, the system employs a high-precision ball screw drive structure. Compared to belt drives, ball screw drives possess extremely high axial stiffness and self-locking capability, enabling the conversion of the stepper motor's rotational motion into precise pallet displacement. Through the ball screw's deceleration and force amplification effect, the system not only improves positioning resolution but also ensures that the pallet does not accumulate positional deviations during prolonged repetitive operations, providing a reliable coordinate reference for the multi-directional vision acquisition system. This drive structure, in conjunction with the PUL and DIR signals output from the aforementioned STM32 control board, achieves precise digital control of the pallet's running speed and stroke.

[0166] The structural design and surface treatment of the tray itself are crucial for ensuring high-quality imaging. Considering the reflective nature of the lock's painted surface and its extreme sensitivity to surface scratches, the tray surface of this system is specially coated with custom-made matte frosted paper. This material selection has multiple engineering implications: First, the diffuse reflection properties of the matte surface absorb excess stray light, effectively preventing background reflection from interfering with the lock edge defect recognition algorithm and significantly improving the image signal-to-noise ratio. Second, the frosted paper has a certain degree of flexibility, acting as a cushioning layer between the lock and the metal tray, effectively preventing damage caused by rigid contact during loading and unloading. Finally, the frosted material greatly increases the coefficient of friction of the contact surface, solving the problem of inertial displacement caused by frequent starts and stops when the lock moves in and out of the acquisition area with the tray. This design ensures the lock's pose remains fixed throughout the detection process, providing a stable and consistent guarantee for the ROI extraction of the visual algorithm.

[0167] 3.4 Terminal Embedded Edge AI Hardware Platform

[0168] In visual inspection systems, the efficiency of defect recognition algorithms and the throughput of image data are directly limited by the computing power and architecture of the core computing platform. This project requires high-resolution, real-time deep learning inference to detect minute defects such as particles and scratches on the painted surfaces of locks. Traditional low-computing-power embedded chips cannot meet the timeliness requirements. Therefore, the system builds a terminal embedded edge AI hardware platform with an edge computing chip at its core, a high-resolution industrial camera as the visual source, and a large-size touchscreen as the interaction medium. Its overall architecture and peripheral topology provide computing power support for the system's efficient operation.

[0169] The system uses the NVIDIA Jetson Orin NX as its central edge computing platform. Based on the NVIDIA Ampere architecture GPU, this platform integrates 1024 CUDA cores and 32 Tensor cores, and is equipped with an 8-core ARM Cortex-A78AE CPU, providing up to 70 TOPS of INT8 sparse computing power. Compared to its predecessor, the Orin NX maintains a compact size and low power consumption while offering nearly 5 times the AI ​​inference performance, perfectly capable of running the lightweight deep convolutional neural network deployed in this project.

[0170] At the hardware interface design level, Orin NX provides a rich set of industrial peripheral interfaces through the Carrier Board. Among them, the high-speed UART pin interfaces with the aforementioned STM32 underlying control board to achieve millisecond-level asynchronous bidirectional communication between control commands and hardware status; the built-in PCIe Gen4 channel and USB 3.2 Gen2 interface ensure the parallel processing of multiple high-bandwidth data streams, effectively eliminating the latency bottleneck between image transmission and algorithm processing.

[0171] To convert the physical characteristics of surface defects in locks into high-quality digital signals, this system uses the North Silver Fox U1803 industrial camera. This camera is equipped with a high-sensitivity CMOS image sensor with an ultra-high resolution of 18 megapixels, capable of accurately capturing micron-level scratches and bubbles on painted surfaces. The camera uses a USB 3.0 high-speed data transfer interface and supports the MIICAM protocol, providing a stable, high-frame-rate raw image stream to the Jetson Orin NX platform while ensuring lossless data transmission.

[0172] As the first barrier in visual imaging, the focal length and distortion rate of the optical lens directly affect the accuracy of ROI extraction. This system is equipped with a 4mm fixed-focus lens for the camera. Due to the compact space of the lock inspection station, the wide angle-of-view characteristic of the 4mm lens allows the camera to completely cover the visual acquisition area of ​​the entire electric tray at a relatively short object distance. At the same time, the lens has high resolution and ultra-low distortion rate. Combined with the coaxial diffuse reflection light source designed above, it can uniformly receive the surface light source after multiple diffusions, ensuring the sharpness of the edge pixels of the image and overcoming the barrel or pincushion distortion caused by curved surface reflection to the greatest extent. This provides a high-fidelity feature source for the upper-level edge AI algorithm.

[0173] The human-machine interface (HMI) is a crucial component of the localized deployment and equipment commissioning of the inspection system. This system integrates a 10.1-inch high-definition touchscreen display with a physical resolution of 1280×800. It uses a standard HDMI interface for point-to-point digital communication with the Jetson Orin NX display output, ensuring that the rendered images output by the algorithm are presented to on-site operators losslessly and in real-time. The display integrates a high-sensitivity capacitive multi-touch panel, transmitting touch coordinate signals back to the Orin NX system via a USB interface. In the high electromagnetic interference environment of industrial settings, this touchscreen exhibits excellent anti-static and noise reduction performance. Through a GUI interface developed under the Linux system, inspection personnel can perform one-click calibration, threshold adjustment, inspection mode switching, and local retrieval of historical defect data, significantly improving the integration and independent on-site operation capability of the entire vision inspection system.

[0174] 3.5 Terminal Casing and Internal Structure Design

[0175] The equipment casing, as the physical carrier of the entire lock inspection system, directly affects the stability of the light field, the security of the internal circuitry, and the degree of automation in the inspection process due to its structural design. This is particularly important considering the high requirements for ambient light shielding and mechanical integration in defect detection, such as... Figure 8 As shown, the system's tooling enclosure adopts a three-dimensional, double-layered partitioned structure design. This hierarchical layout not only achieves effective physical isolation between strong and weak currents but also greatly optimizes the equipment's maintenance space, ensuring stable operation of visual inspection tasks in a controlled, enclosed environment. Figure 9 As shown, the upper area is designed as a separate device placement compartment, primarily used to integrate the system's "brain" and "blood vessels." The main control development board, edge computing platform, and power module are all mounted in this space using fixed supports. This layered design allows for neat and controlled internal cabling, effectively preventing stray cables from interfering with the visual imaging area. It also utilizes the top ventilation and heat dissipation structure to provide a good thermal management environment for the high-performance computing units. Furthermore, the upper compartment also includes necessary external interface panels for convenient system debugging and power connection.

[0176] The lower area is defined as the core inspection work area, the center of the vision acquisition system. This area employs strict light-blocking to eliminate the influence of ambient light on defect identification. Five industrial cameras and corresponding light sources are installed around the area. This five-camera surround configuration provides coverage of the lock from top and four side views, ensuring the detection of even minor defects on complex workpiece surfaces. An electric tray entry / exit track is designed at the bottom of the work area. A stepper motor drives the tray back and forth between the loading and inspection positions via a lead screw drive. Once the tray carrying the lock enters the inspection area and is positioned, the five cameras trigger acquisition, laying the mechanical foundation for automated detection of lock paint defects. The top and front views of the structure are shown below. Figure 10 As shown.

[0177] This section completes the hardware platform construction of the lock defect detection system. Through a self-designed STM32 control board, analog pressure-controlled coaxial light source, and precision electric tray, the system solves the imaging and positioning challenges of highly reflective objects in complex environments. The overall structure is rationally partitioned, and the hardware and software work together stably, providing a high-quality data acquisition environment for the subsequent deployment of deep learning algorithms.

[0178] This invention establishes a multi-directional image acquisition environment for highly reflective painted surfaces and constructs a high-quality defect dataset. Addressing issues such as indistinct lock defect features and specular reflection interference from painted surfaces, this invention independently designs and implements a dedicated coaxial vision lighting system. Utilizing a soft light multi-diffusion mechanism, it transforms point-like intense light into a uniform surface light source, minimizing "glare" on highly reflective painted surfaces and significantly improving the feature contrast of minute defects. Simultaneously, multiple high-definition industrial cameras are scientifically arranged around the inspection chamber, acquiring image data from five directions of the lock from all angles through a time-division polling trigger mechanism, ensuring comprehensive surface feature capture. In the data preprocessing stage, key regions are extracted through ROI division, and multi-dimensional dynamic expansion is performed using data augmentation techniques such as rotation, flipping, and brightness adjustment. This effectively alleviates the problem of uneven sample distribution in industrial settings, laying a solid foundation for the high robustness of subsequent algorithms.

[0179] This invention makes targeted improvements to general object detection networks, designing and training a high-precision model for lock classification and minor defect recognition. The invention proposes an algorithm architecture based on an improved YOLO26: In the lock classification stage, a novel WTConv convolutional module is introduced, utilizing wavelet transform to decompose features into different frequency bands while preserving spatial resolution, achieving a large receptive field without significantly increasing the number of parameters, thus greatly improving the model's ability to perceive the global context structure; In the minor defect recognition stage, for targets with extremely small geometric scales such as tiny bubbles, particles, and scratches, an SPD-Conv transform convolutional module is introduced to completely replace the traditional stride convolution and pooling layers. Through a space-to-depth transformation, spatial information is losslessly transferred to the channel dimension, avoiding the loss of fine-grained information and significantly enhancing the network's feature extraction capability and sensitivity to low-resolution and minor defects. The trained and optimized model is deeply deployed on an embedded edge computing platform, and collaborative testing and analysis of the overall system functionality and performance are conducted. The system hardware utilizes the NVIDIA Jetson Orin NX as its core computing platform, employing the TensorRT inference optimization engine for layer fusion and half-precision quantization acceleration. This significantly reduces inference latency and memory usage with minimal loss of accuracy. The lower-level machine uses an STM32 microcontroller as its control core, with embedded programs developed based on the HAL library. It employs a precise PID algorithm for closed-loop control of the light source brightness and a high-precision ball screw drive structure to power a 57-stepper motor-driven electric tray, enabling smooth entry and exit of locks from the inspection chamber and precise spatial positioning. Experimental results demonstrate that the system operates stably in both real and simulated industrial environments, achieving an accuracy rate of over 95% in identifying minor surface defects on locks. Both the detection speed and positioning accuracy meet the standards for quality inspection in industrial production lines, successfully realizing a closed-stack, fully automated inspection process from perception and decision-making to feedback.

Claims

1. A method for detecting minor defects in lock paint using multi-directional visual edge calculation, characterized in that, Based on the lock painting minor defect detection system, the lock painting minor defect detection system includes an STM32 control board, an edge computing platform, an electric tray device, an industrial camera, a lighting source and an external interface panel. Includes the following steps: Step S1: The STM32 control board receives instructions from the edge computing platform, controls the electric pallet device to transport the lock to the inspection station, and triggers the five industrial cameras and coaxial lighting source to collect raw images of the lock from five directions. Step S2: The edge computing platform divides the acquired image into ROIs, extracts the main lock area, and performs data augmentation processing; Step S3: Call the preset lock classification model to perform inference on the main view image and output the lock model; the lock classification model is a network based on the improved YOLO26-WTConv, in which the WTConv module uses discrete wavelet transform to perform multi-level frequency decomposition and reconstruction of the feature map to expand the receptive field; Step S4: Call the preset defect detection model to perform batch inference on the five images, and output the category, location coordinates and confidence of defects in each image; the defect detection model is a network based on the improved YOLO26-SPD-Conv, in which the SPD-Conv module achieves lossless downsampling through spatial-to-depth transformation to preserve the features of minor defects; Step S5: Based on the reasoning results of steps S3 and S4, determine whether the lock is qualified and display the test results on the external interface panel in real time.

2. The method for detecting minor defects in lock paint using multi-directional visual edge calculation according to claim 1, characterized in that, The ROI partitioning method in step S2 is as follows: S21: Obtain the spatial activation map: During the forward inference process of the lock classification model on the main view image, extract the feature response map output by the last convolutional layer in the classification branch. The spatial dimension of this response map is 1 / 32 of the original image. S22: Generate activation mask: Convert the response map into a probability distribution using the Softmax function, select the activation map corresponding to the channel with the highest probability value as the saliency map, and perform bilinear upsampling of the saliency map to the original image size; S23: Locating connected regions: Adaptive threshold segmentation is applied to the upsampled saliency map to obtain a binary mask, and then the connected region with the largest area in the mask is extracted as the candidate region of the lock body; S24: Region Expansion: Based on the minimum bounding rectangle of the candidate region, expand the width and height of the original rectangle by 10% in each of the four directions of upward, downward, left and right to obtain the final ROI bounding box; S25: Cropping Output: Crops the main lock area from the original image based on the ROI bounding box, and scales the cropped image to the standard input size of the defect detection model.

3. The method for detecting minor defects in lock paint using multi-directional visual edge calculation according to claim 1, characterized in that, The lock classification model in step S3 is trained using the following method: S31: Constructing the basic network: Using YOLO26-cls as the baseline classification network, the YOLO26-cls network includes a backbone network, a classification head, and a full pipeline aggregation and distribution structure; S32: Embedding WTConv modules: Within the C3k2 Bottleneck structure of the high-level feature extraction layer of the backbone network, some standard 3×3 depth convolutional layers are replaced with WTConv modules; S33: Forward computation of the WTConv module: Apply two-dimensional Haar wavelet transform to the input feature map X, decompose it into four frequency band components: low-frequency approximate component, horizontal high-frequency component, vertical high-frequency component, and diagonal high-frequency component; The wavelet decomposition above is recursively performed on the low-frequency components, with a total of 2 to 3 levels of decomposition. After each level of decomposition, the spatial resolution is halved, and the receptive field is exponentially expanded. At each decomposition level, a 3×3 deep convolutional layer is applied to the four frequency band components with a stride of 1, without changing the size of each component. The feature map is reconstructed step by step from the deepest level using inverse discrete wavelet transform, and the reconstruction result is weighted and aggregated with the feature map of the previous level. S34: Training configuration: cross-entropy loss function is used, AdamW optimizer is used, and cosine annealing decay strategy is adopted. The training rounds are 600, the input image size is kept at high resolution, and supervised training is performed with lock model as the unique label. S35: Model Output: After training, the model can output the predicted probability distribution of the lock model, which is used for real-time classification reasoning in step S3.

4. The method for detecting minor defects in lock paint using multi-directional visual edge calculation according to claim 1, characterized in that, The defect detection model in step S4 is trained using the following method: S41: Constructing the basic network: The YOLO26-P2 network is used as the benchmark detection network. This network contains four high-resolution feature detection heads with 4x downsampling, 8x downsampling, 16x downsampling and 32x downsampling, and has a feature pyramid network and path aggregation network structure. S42: Replace downsampling layers: Replace all traditional convolutional layers and max pooling layers with stride 2 in the backbone network and feature fusion stage with SPD-Conv modules. S43: Forward computation of the SPD-Conv module: Assume the input feature map size is H×W×C; From spatial to deep layers: The input feature map is divided into four sub-feature maps based on the parity of the pixel coordinates, with each sub-map having a size of [size missing]. Then, the features are concatenated along the channel dimension to obtain an intermediate feature map of size [size missing]. ; Non-strut convolutional layer: Apply a standard convolutional layer with a kernel size of 3×3 and a stride of 1 to the intermediate feature map output by the SPD layer to reduce the number of channels from 4C to the preset number of output channels. This convolutional layer does not further reduce the spatial resolution. S44: Construct a composite loss function: The bounding box regression loss uses CIoU loss, the classification loss uses binary cross-entropy loss, and the confidence loss also uses binary cross-entropy loss. The total loss is the weighted sum of CIoU loss, binary cross-entropy loss, and binary cross-entropy loss. S45: Training configuration: SGD optimizer is used, with an initial learning rate of 0.01, momentum of 0.937, and weight decay of 0.0005. Mosaic data augmentation and MixUp strategies are enabled during training, and the training epochs are 600. S46: Model Output: After training, the model can output bounding box coordinates, category labels, and confidence scores for defects such as scratches, dents, paint particles, and stains from the input image.

5. The method for detecting minor defects in lock paint using multi-directional visual edge calculation according to claim 1, characterized in that, The lock classification model and defect detection model employ a slice and whole-image incremental learning strategy during the training phase. The first stage: The high-resolution original image is cut into 1024×1024 pixel sub-patterns according to a set overlap ratio, and the model is trained using the sub-patterns; Second stage: Load the pre-trained weights obtained from the first stage of training, and use the complete original large image to perform secondary fine-tuning of the model.

6. The method for detecting minor defects in lock paint using multi-directional visual edge calculation according to claim 1, characterized in that, The inference process in steps S3 and S4 is accelerated using TensorRT: the trained PyTorch model is converted into the TensorRT engine format, layer fusion, automatic kernel tuning and FP16 half-precision quantization are performed, and then it is deployed on an edge computing platform.

7. A lock painting minor defect detection system based on multi-directional visual edge computing, characterized in that, A method for detecting minor defects in lock paint using multi-directional visual edge calculation as described in any one of claims 1-6, comprising: A three-dimensional, double-layered shell, with the interior horizontally divided into an upper and lower area; An embedded control platform component is installed in the upper region, including an STM32 control board, an edge computing platform, and a power module; A core detection work area is located in the lower layer area. This area has a closed, light-shielding structure and includes: An electric tray unit is fixed to the bottom of the lower area; At least five sets of industrial cameras are arranged around the inspection station of the electric pallet device, including one top-view camera and at least four side-view cameras; Multiple sets of coaxial illumination sources, each coaxial illumination source is installed at the front end of an industrial camera, and the optical axis of the light source is coaxial with the imaging axis of the camera.

8. The lock painting minor defect detection system based on multi-directional visual edge calculation according to claim 7, characterized in that, The upper and lower regions of the three-dimensional double-layer partitioned shell are physically isolated by a metal partition. The metal partition is provided with wire-passing holes, and a shielding ring for electromagnetic interference is provided at the wire-passing holes. The STM32 control board and edge computing platform are suspended on the metal partition of the upper region by multiple metal fixed pillars, and an air convection gap is left between them and the metal partition. An external interface panel is provided on the rear or side wall of the upper region. The external interface panel integrates a power input interface, a debugging serial port, a USB interface, and an HDMI display interface.

9. The lock painting minor defect detection system based on multi-directional visual edge calculation according to claim 7, characterized in that: The electric tray device includes a stepper motor, a high-precision lead screw, and a carrying tray; the upper surface of the carrying tray is covered with matte frosted paper; the bottom of the carrying tray is provided with a nut seat that cooperates with the lead screw, and the two sides of the carrying tray are provided with sliders that slide in cooperation with the bottom guide rail of the lower area.

10. The lock painting minor defect detection system based on multi-directional visual edge calculation according to claim 7, characterized in that: The coaxial lighting source includes a white mist LED light panel, a diffused light paper layer, and a light source housing; the light source housing is fixedly nested in front of the top-view camera; the four side-view cameras are arranged in a 90° interval ring around the detection station of the electric tray, and the optical axes of the cameras all point to the center of the tray.

Images