Industrial complex scene small target defect detection system based on attention mechanism

Abstract

This invention discloses an attention-based system for detecting small-target defects in complex industrial scenes, belonging to the field of industrial vision inspection technology. It includes modules for image acquisition, preprocessing, feature perception, candidate recognition, decision feedback, and localization optimization. The image acquisition module acquires a set of original scene images of the workpiece; the preprocessing module eliminates uneven illumination to generate standard illumination image data; the feature perception module calculates a global context association matrix through a multi-level feature perception network; the candidate recognition module parses the matrix to identify candidate defect regions and extracts local texture gradient information to construct a defect probability distribution heatmap; the decision feedback module compares the results with a sample library, outputs the decision results, and feeds back to the network to update weights; and the localization optimization module recalculates the association matrix to generate optimized defect location coordinates. This invention can improve the detection and localization accuracy of small-target defects in complex industrial scenes, adapting to the actual inspection needs of industrial sites.

Description

Technical Field

[0001] This invention belongs to the field of industrial visual inspection technology, specifically an attention-based defect detection system for small targets in complex industrial scenes. Background Technology

[0002] In complex industrial environments, small-target defect detection typically employs panoramic vision acquisition devices to obtain raw scene images of the workpiece to be inspected. Defect feature extraction and recognition are then performed directly based on these raw images. Image data analysis is completed using a single-level feature extraction network, and defect determination and location are achieved through a single feature matching operation. Defect region identification is achieved only through basic feature screening, without specialized, refined feature processing and accurate identification for small-target defects. Furthermore, existing detection methods do not perform uneven illumination correction on the raw images. Differences in lighting conditions in industrial settings directly cause image data feature distortion, making it impossible to generate standardized illumination image data and affecting the accuracy of subsequent defect feature analysis.

[0003] Existing network structures used in defect detection cannot calculate global contextual feature relationships, lack feedback iteration mechanisms for defect judgment results, and cannot adaptively adjust and update network weights based on detection results. Local texture gradient information in small target defect regions cannot be effectively extracted, defect probability distribution cannot be presented in a quantified form, and defect localization relies solely on single feature calculations, resulting in biased localization results. This invention aims to eliminate the interference of industrial lighting on image data, generate standard illumination image data, calculate the global contextual relationship matrix through a multi-level feature perception network, construct a closed-loop feedback mechanism for defect judgment results, extract local texture gradient information of small target defects to construct a defect probability distribution heatmap, and optimize defect localization coordinates after iterative updates of network weights. This addresses the problems of insufficient detection accuracy and localization bias in small target defect detection in complex industrial scenarios. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; To address this, the present invention proposes a small target defect detection system based on an attention mechanism in complex industrial scenarios, comprising: The image acquisition module is equipped with a panoramic vision acquisition device for the industrial site to acquire the original scene image set of the workpiece to be inspected; The preprocessing module performs illumination unevenness elimination processing on the original scene image set to generate standard illumination image data; The feature perception module inputs the standard illumination image data into a multi-level feature perception network and calculates the global context association matrix; The candidate identification module parses the global context association matrix, identifies potential small target defect regions, marks them as high-interest candidate regions, extracts the local texture gradient information of the high-interest candidate regions, and constructs a defect probability distribution heatmap. The judgment feedback module compares the similarity index of the defect probability distribution heatmap with the preset defect sample library and outputs the preliminary defect judgment result. The preliminary defect determination result is fed back to the multi-level feature perception network, triggering iterative updates of the network's internal weights. The localization optimization module, after completing the iterative update of the internal weights of the network, recalculates the global context association matrix and generates optimized defect localization coordinates.

[0005] Furthermore, the configuration of the industrial site panoramic vision acquisition device to acquire the original scene image set of the workpiece to be inspected includes: Multiple heterogeneous vision sensors are arranged along the conveying direction of the industrial production line to simultaneously capture the front view, side view, and top view of the workpiece to be inspected. Record the acquisition time and workpiece identification code of the front view, side view and top view images to form an image data packet with time sequence attributes; Spatial registration is performed on the image data packets to eliminate geometric distortions between images from different viewpoints and integrate them to generate an original scene image set of the workpiece to be inspected, specifically including: Extract the workpiece identification code feature points from the front view image, side view image and top view image, and locate at least four non-collinear physical marker points in each view image; Based on the pixel coordinates of the physical markers in the images from different viewpoints, the perspective transformation matrix between the images from different viewpoints is calculated, and the perspective transformation matrix is ​​used to perform geometric projection correction on the images from different viewpoints to eliminate geometric distortion caused by differences in shooting angles. Based on the time sequence attributes of the acquisition time, the images of each view after geometric projection correction are time-series aligned, and the multi-view corrected images corresponding to the same workpiece identification code are fused according to the preset spatial stitching rules to generate an original scene image set of the workpiece to be inspected containing complete surface information.

[0006] Further, performing illumination unevenness elimination processing on the original scene image set to generate standard illumination image data includes: Analyze the brightness distribution histogram of each pixel in the original scene image set to identify non-uniform lighting areas with strong light reflection or shadow occlusion; Based on the location information of the non-uniform illumination area, an adaptive brightness compensation template is constructed; Traverse all pixels of the original scene image set, adjust the grayscale value of each pixel according to the adaptive brightness compensation template, and output standard illumination image data. The step of constructing an adaptive brightness compensation template based on the location information of the non-uniform illumination region includes: A morphological dilation operation is performed on the non-uniform illumination region to expand the region boundary to cover the transition illumination band, generating a composite illumination mask that includes a core illumination anomaly region and a transition region. Within the composite illumination mask, a 5×5 pixel local neighborhood window is divided with each pixel as the center. The mean and standard deviation of the pixel grayscale within the window are calculated as the local illumination statistics of that pixel. Based on the local illumination statistics, a bicubic interpolation algorithm is used to fill in the missing brightness values ​​in the non-uniform illumination area to generate an initial brightness compensation matrix. A Gaussian smoothing filter is introduced to convolve the initial brightness compensation matrix to filter out high-frequency noise interference, resulting in a smoothed adaptive brightness compensation template.

[0007] Further, the standard illumination image data is input into a multi-level feature perception network, and the calculation of the global context association matrix includes: The standard illumination image data is divided into several overlapping image blocks, and the edge contour features and color distribution features of each image block are extracted respectively; The edge contour features and color distribution features are mapped to a high-dimensional semantic space to generate a set of multi-dimensional feature vectors. Calculate the semantic distance between any two vectors in the multidimensional feature vector set, and construct a global context association matrix reflecting global dependencies based on the semantic distance.

[0008] Furthermore, parsing the global context correlation matrix identifies potential small target defect regions, which are then marked as high-interest candidate regions, including: Scan the non-zero element clusters in the global context association matrix to locate the coordinates of feature mutations; Statistically analyze the feature density values ​​within a preset neighborhood around the coordinate position, and filter out regions where the feature density values ​​exceed a dynamic threshold. The selected regions are cropped from the standard illumination image data and marked as high-interest candidate regions.

[0009] Further, extracting the local texture gradient information of the high-interest candidate region and constructing a defect probability distribution heatmap includes: Perform multi-directional gradient operator operations on the high-interest candidate region to obtain the horizontal gradient magnitude and the vertical gradient magnitude; The horizontal and vertical gradient magnitudes are combined to calculate the overall texture complexity index. The comprehensive texture complexity index is mapped back to the corresponding pixel position of the high-attention candidate region to generate a heatmap of defect probability distribution with probability values.

[0010] Furthermore, by comparing the similarity index between the defect probability distribution heatmap and the preset defect sample library, the preliminary defect judgment result is output, including: Retrieve historical defect images stored in the preset defect sample library and extract the baseline heatmap features of the historical defect images; Calculate the shape and area parameters of high-probability regions in the defect probability distribution heatmap and match them with the features of the baseline heatmap; Based on the matching score, the workpiece to be inspected is classified as a suspected defective part or a normal part, and the classification result is recorded as a preliminary defect judgment result.

[0011] Furthermore, feeding back the preliminary defect determination result to the multi-level feature perception network and triggering iterative updates of the network's internal weights includes: Analyze the missed areas that were misjudged as normal parts in the preliminary defect determination results, and extract the original feature data of the missed areas; Calculate the difference loss value between the original feature data of the missed detection region and the output features of the current multi-level feature perception network; The difference loss value is backpropagated to the hidden layer of the multi-level feature perception network to adjust the weight parameters of the convolution kernel, thus completing one internal weight iteration update of the network.

[0012] Furthermore, after completing the iterative update of the network's internal weights, the global context association matrix is ​​recalculated to generate optimized defect location coordinates, including: After loading the multi-level feature perception network with iterative weight updates within the network, the standard illumination image data is input again. Repeat the feature extraction and association matrix construction process to obtain the corrected global context association matrix; The coordinate information of the maximum response point is extracted from the modified global context association matrix and converted into optimized defect location coordinates in the industrial coordinate system.

[0013] Furthermore, the method also includes: Collect real-time environmental dust concentration data at industrial sites and monitor the cleanliness of the vision sensor lens; When the real-time environmental dust concentration data exceeds the warning line or the lens cleanliness status is lower than the standard value, the filter intensity coefficient of the adaptive brightness compensation template is automatically adjusted. The adjusted filter intensity coefficients are applied to the processing of the original scene image set to regenerate standard illumination image data.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The original scene image set undergoes illumination unevenness elimination processing to generate standard illuminance image data. This standard illuminance image data is then input into a multi-level feature perception network to calculate the global context association matrix. Preliminary defect judgment results are fed back to the multi-level feature perception network, triggering iterative updates of the network's internal weights. Illumination unevenness elimination processing removes image data interference caused by variations in industrial lighting conditions, ensuring consistent illuminance standards and guaranteeing the stability of the foundational data for subsequent feature analysis. The multi-level feature perception network enables comprehensive mining of global feature relationships within the image, overcoming the limitations of single-level networks in feature extraction. The global context association matrix fully reflects the association status of features in each region of the image. The closed-loop feedback of defect judgment results drives adaptive adjustment of the network's internal weights based on actual detection results, allowing the network's feature extraction parameters to match the image feature attributes of complex industrial scenes, ensuring consistency between the calculated global context association matrix and the actual defect feature distribution of the workpiece.

[0015] The global context association matrix (GIM) is analyzed to identify potential small target defect regions and mark them as high-interest candidate regions. Local texture gradient information is extracted from these high-interest candidate regions, and a defect probability distribution heatmap is constructed. After weight iteration updates in a multi-level feature perception network, the GIM is recalculated to generate optimized defect location coordinates. Local texture gradient information enhances the detailed feature representation of small target defects, highlighting the feature differences between small target defects and normal areas. The defect probability distribution heatmap quantifies the probability of defect presence, intuitively reflecting the distribution of defects in the image. The recalculated GIM corrects deviations in the previous feature extraction and analysis process, refines the coordinate parameters for defect location, reduces the error between the location result and the actual defect location, and ensures that the defect location result closely matches the actual distribution of small target defects in complex industrial scenarios, meeting the technical requirements for accurate small target defect location. Attached Figure Description

[0016] Figure 1 This is a timing diagram of the attention mechanism-based small target defect detection system for complex industrial scenarios described in this invention. Figure 2 A flowchart for performing illumination unevenness elimination processing on the original scene image set to generate standard illumination image data; Figure 3 A path planning diagram for PCB defect detection in an industrial production line; Figure 4 Performance trends of multiple batches of solar cells were tested; Figure 5 Box plot comparing defect location accuracy before and after weight iteration optimization. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 This invention provides an attention-based system for detecting small-target defects in complex industrial scenes. The system includes: after system startup, multiple panoramic vision acquisition devices configured in the industrial site acquire a set of original scene images of the workpiece to be inspected via an image acquisition module. A preprocessing module performs illumination unevenness elimination processing on the original scene image set to generate standard illuminance image data with uniform illumination. This standard illuminance image data is input into a multi-level feature perception network within a feature perception module, which calculates and outputs a global contextual association matrix reflecting the global semantic dependencies of the image. A candidate recognition module parses this global contextual association matrix, identifies potential small defect regions, marks these regions as high-attention candidate regions, extracts the local texture gradient information of each high-attention candidate region, and constructs a defect probability distribution heatmap based on this information. A judgment feedback module compares the defect probability distribution heatmap with the sample features stored in a preset defect sample library for similarity, and outputs a preliminary defect judgment result based on the similarity index obtained from the comparison. This preliminary defect judgment result is fed back to the multi-level feature perception network, which calculates the loss based on this result and triggers iterative updates of its internal weight parameters. After the weights are updated, the localization optimization module controls the multi-level feature perception network to reprocess the standard illumination image data, calculates a new and optimized global context association matrix, and generates accurate optimized defect location coordinates based on this matrix, thus completing a full detection process.

[0019] In one embodiment of the present invention, in a specific implementation in an industrial setting, the image acquisition module employs multiple heterogeneous vision sensors with different imaging characteristics arranged along the transport direction of the industrial assembly line. These sensors are synchronously triggered and can capture frontal, side, and top views of the same workpiece from different angles. The system records the precise time of acquisition for each image and its corresponding workpiece identification code, forming a set of image data packets with strict time-series attributes. Spatial registration is performed on the image data packets. This operation first extracts feature points composed of workpiece identification codes or preset marker points from the frontal, side, and top views, and locates at least four non-collinear physical marker points in each view image. Based on the pixel coordinates of these physical marker points in their respective view images, the system calculates the perspective transformation matrix from the side and top views to the frontal view. Using the calculated perspective transformation matrix, geometric projection correction is performed on each view image to eliminate geometric distortion introduced by differences in shooting angles. Based on the time-series attributes of the acquisition time, the system performs temporal alignment on the geometrically corrected view images to ensure that images of the same workpiece captured at different times can accurately correspond. The multi-view images corresponding to the same workpiece identification code, after correction and alignment, are fused pixel-level according to a preset spatial stitching rule to finally generate an original scene image set containing complete surface information of the workpiece to be inspected.

[0020] In specific implementations, three heterogeneous vision sensors are arranged along the conveyor direction of the automotive engine block assembly line. These sensors include a front-facing high-resolution camera, a side-facing infrared camera, and a top-facing multispectral camera. The sensors simultaneously capture front, side, and top views of the same engine block. Specifically, the front-facing high-resolution camera captures the front view to obtain texture details of the cylinder block's front, the side-facing infrared camera captures the side view to obtain thermal distribution information on the cylinder block's side, and the top-facing multispectral camera captures the top view to obtain the spectral characteristics of the material on the top of the cylinder block. In some embodiments, the system records the acquisition time of the front, side, and top views along with a two-dimensional workpiece identification code laser-etched on the engine block. The acquisition time is accurate to the millisecond level and synchronized with the assembly line cycle. The two-dimensional workpiece identification code uniquely associates the engine block's production batch and serial number, forming an image data package with strict time-series attributes. In practical implementation, spatial registration is performed on the image data packets. This spatial registration operation first extracts the corner features of the two-dimensional workpiece identification code from the front view, side view, and top view images, along with the features of four physical marker points pre-set on the engine block fixture. These physical marker points are made of highly reflective material and appear as clear circular spots in each view image. In practical implementation, the perspective transformation matrix between images from different viewpoints is calculated based on the pixel coordinates of the physical marker points in each view image. The perspective transformation matrix maps the side view and top view images to the coordinate system of the front view image. The calculation of the perspective transformation matrix is ​​achieved by solving the following system of linear equations:

[0021] in: This represents the homogeneous pixel coordinates of a physical marker point in the side view image. The matrix represents the homogeneous pixel coordinates of the corresponding physical marker points in the front view image. to The perspective transformation parameters are to be determined. The complete perspective transformation matrix is ​​obtained by solving for the coordinates of at least four pairs of non-collinear marker points using the least squares method. In specific implementations, the obtained perspective transformation matrix is ​​used to perform geometric projection correction on the images of each viewpoint. Geometric projection correction resamples and maps the pixels in the side view image and the top view image to eliminate geometric distortions introduced by differences in shooting angles. In specific implementations, after geometric projection correction, the originally tilted cylinder block edge in the side view image is aligned with the edge in the front view image in the front view coordinate system. Optionally, the system performs temporal alignment on the images of each viewpoint after geometric projection correction based on the time sequence attributes of the acquisition time. Temporal alignment ensures that image frames of the same engine cylinder block captured by different heterogeneous vision sensors during the movement of the assembly line are accurately matched on the time axis. Temporal alignment compensates for image sequence offsets caused by small differences in sensor exposure time through interpolation. In some embodiments, multi-view corrected images corresponding to the same two-dimensional workpiece identification code are fused according to a preset spatial stitching rule. The spatial stitching rule places the front view image in the center, and the side view and top view images are scaled and stitched together on both sides to generate an original scene image set containing complete surface information of the engine block. It can be understood that, in terms of data comparison, the same physical marker point in the original multi-view images without spatial registration has a significant offset in pixel coordinates, with the offset ranging from tens of pixels. After perspective transformation matrix correction and temporal alignment, the coordinate error of the same marker point in the multi-view images is reduced to the sub-pixel level, and the fused original scene image set exhibits consistent spatial continuity. Optionally, in another specific implementation, for the aluminum alloy wheel hub inspection scenario, heterogeneous vision sensors are arranged on the wheel hub painting line to capture the front, side, and top view images of the wheel hub. Physical marker points are placed on the wheel hub clamping mechanism, and the original scene image set generated after spatial registration covers the entire outer surface of the wheel hub. In a specific implementation, the acquisition time and workpiece identification code information attached to each image in the image data package are stored in a structured database for subsequent traceability and batch analysis. It is understandable that the generation of the original scene image set depends on accurate geometric projection correction and temporal alignment, and the parameter accuracy of the perspective transformation matrix directly determines the quality of multi-view image fusion.

[0022] In one embodiment of the present invention, when the preprocessing module performs the uneven illumination elimination process, refer to... Figure 2The system analyzes the brightness distribution histogram of each pixel in the original scene image set, identifying non-uniform illumination regions with strong light reflection or shadow occlusion by analyzing the histogram distribution pattern. Based on the location information of the identified non-uniform illumination regions, an adaptive brightness compensation template is constructed. The construction process first performs a morphological dilation operation on the non-uniform illumination regions, expanding their boundaries to cover the transitional illumination bands where the light intensity gradually changes, thereby generating a composite illumination mask that simultaneously contains the core illumination anomaly region and the transition region. Within the area defined by this composite illumination mask, a 5×5 pixel local neighborhood window is divided with each pixel as the center. The mean and standard deviation of the gray values ​​of all pixels within this window are calculated, and these mean and standard deviation are used as the local illumination statistics for that pixel. Based on the local illumination statistics of all pixels, a bicubic interpolation algorithm is used to smoothly fill in the missing brightness values ​​in the non-uniform illumination regions, generating an initial brightness compensation matrix. A Gaussian smoothing filter is introduced to convolve the initial brightness compensation matrix, filtering out high-frequency noise interference that may be introduced by interpolation, and finally obtaining a smoothed adaptive brightness compensation template. The system traverses all pixels of the original scene image set, dynamically adjusts the gray value of each pixel according to the compensation coefficients provided by the adaptive brightness compensation template at the corresponding positions, and outputs a set of standardized illumination image data with normalized lighting conditions.

[0023] In practical implementation, the attention-based industrial complex scene small target defect detection system implements a preprocessing module in the LCD panel detection scenario. The LCD panel surface is prone to high-brightness bands formed by ambient light reflection and shadows caused by equipment obstruction. The preprocessing module performs illumination unevenness elimination processing on the original scene image set of the LCD panel from the image acquisition module. The processing first analyzes the brightness distribution histogram of each pixel in the original scene image set. The brightness distribution histogram counts the frequency of each gray level in the image. In practical implementation, by identifying whether there are bimodal or multimodal shapes and long-tailed distributions in the brightness distribution histogram, the system locates the overly bright areas with strong light reflection and the overly dark areas with shadow obstruction in the image. These areas are collectively referred to as non-uniform illumination areas. In some embodiments, based on the location information of the identified non-uniform illumination region, i.e., the set of pixel coordinates of the region in the image, the system begins to construct an adaptive brightness compensation template. Constructing the adaptive brightness compensation template first involves performing a morphological dilation operation on the non-uniform illumination region. This morphological dilation operation uses a circular structuring element with a radius of 5 pixels to expand the boundary of the non-uniform illumination region to cover the transitional illumination band where the illumination intensity gradually changes, thereby generating a composite illumination mask that simultaneously includes a core highlight area, a core shadow area, and its surrounding gradual transition area. In a specific implementation, within the complete area defined by the composite illumination mask, a 5×5 pixel local neighborhood window is divided with each pixel as the center. The local neighborhood window calculates the mean and standard deviation of the grayscale values ​​of all 25 pixels within the window. This mean and standard deviation are used as the local illumination statistics for the central pixel. The mean in the local illumination statistics reflects the average brightness level of the window, and the standard deviation reflects the degree of brightness fluctuation within the window.

[0024] Based on the calculated local illumination statistics for each pixel, the system employs a bicubic interpolation algorithm to fill in the missing brightness values ​​in non-uniform illumination regions. The bicubic interpolation algorithm considers the local illumination statistics of 16 neighboring points around a pixel, generating a smooth brightness transition through a cubic polynomial, resulting in an initial brightness compensation matrix with the same size as the original scene image set. The value at each position in the initial brightness compensation matrix... Calculated using the following formula:

[0025] in: This represents the coordinates of the pixel to be interpolated in the initial brightness compensation matrix. Representing coordinates The mean gray level in the local illumination statistics corresponding to neighboring pixels. and This is a weighting function, calculated based on the distance between the interpolation point and its 16 nearest neighbors, with closer neighbors receiving greater weights. In practice, a Gaussian smoothing filter with a standard deviation of 1.5 is introduced to perform two-dimensional convolution on the initial brightness compensation matrix. This Gaussian smoothing filter removes high-frequency noise interference that may be introduced by the bicubic interpolation process, resulting in a smoothed adaptive brightness compensation template. It can be understood that each pixel position in the adaptive brightness compensation template stores a brightness correction coefficient. By traversing all pixels in the original scene image set, and based on the correction coefficients provided by the adaptive brightness compensation template at the corresponding positions, the grayscale values ​​of each pixel in the original scene image set are dynamically adjusted, outputting a set of standardized illumination image data. In some embodiments, data comparison shows that for an original image of a liquid crystal panel with significant highlights, the average grayscale value of pixels in the highlight area is 235, the average grayscale value in the shadow area is 45, and the grayscale standard deviation is 78. After adjustment by the adaptive brightness compensation template, the average grayscale value of pixels in the corresponding area in the generated standard illumination image data is adjusted to 168, and the overall pixel grayscale standard deviation is reduced to 24. Optionally, in metal shell detection scenarios, areas with strong reflective light spots appear as a saturated white in the original image. After the above processing, the texture details of the shell surface are recovered in the standard illumination image data. In specific implementations, the radius parameter of the structuring element in the morphological dilation operation can be configured according to the image resolution. It can be understood that the generation of the adaptive brightness compensation template depends entirely on the local illumination statistics of the image itself, without the need for a preset illumination model. Optionally, for images with relatively uniform illumination conditions, non-uniform illumination areas may be empty. In this case, the generated adaptive brightness compensation template is a matrix with all elements equal to 1, and the grayscale values ​​of the original scene image set will remain unchanged.

[0026] In one embodiment of the present invention, the feature perception module inputs standard illumination image data into its multi-level feature perception network. The network first segments the input standard illumination image data into several image blocks with certain overlapping areas, and extracts the edge contour features and color distribution features of each image block. The network maps the extracted edge contour features and color distribution features to a high-dimensional semantic space, in which each image block is represented as a multi-dimensional feature vector, and the feature vectors of all image blocks constitute a multi-dimensional feature vector set. The network calculates the semantic distance between any two feature vectors in this multi-dimensional feature vector set, which reflects the degree of semantic association between the two image blocks. Based on the calculated semantic distances between all paired vectors, a global context association matrix reflecting the global dependencies between all regions of the entire image is constructed. When parsing the global context association matrix, the candidate recognition module scans the regions where non-zero elements are clustered in the matrix and locates the coordinates where features change abruptly. The feature density values ​​in the preset neighborhood around each coordinate position are counted, and a dynamic threshold is calculated based on the overall feature distribution of the current image to filter out regions where the feature density value exceeds the dynamic threshold. Based on the selected region coordinates, the system crops out the corresponding image region from the original standard illumination image data and marks it as a high-interest candidate region for subsequent detailed analysis.

[0027] In practical implementation, the attention-based industrial complex scene small target defect detection system implements the corresponding steps of the feature perception module and the candidate recognition module in the printed circuit board inspection scenario. In the specific implementation, the feature perception module receives the standard illumination image data of the printed circuit board from the preprocessing module. The standard illumination image data size is 2048x2048 pixels. The multi-level feature perception network inside the feature perception module first divides the input standard illumination image data of the printed circuit board into several overlapping image blocks. The image block size is set to 32x32 pixels, and the overlap area between adjacent image blocks is 8 pixels. In the specific implementation, a total of 3969 image blocks are generated for the entire standard illumination image data of the printed circuit board. The multi-level feature perception network extracts the edge contour features and color distribution features of each image block. The edge contour features are extracted through a three-layer convolutional neural network sub-network, outputting a 128-dimensional feature vector. The color distribution features are calculated by calculating the histogram of the image block in the HSV color space and encoded through a fully connected layer, outputting a 64-dimensional feature vector. In some embodiments, the multi-level feature perception network concatenates the edge contour features and color distribution features of each extracted image patch to form a 192-dimensional joint feature vector. This 192-dimensional joint feature vector is then input into a non-linear mapping layer, which maps it to a 512-dimensional high-dimensional semantic space, generating a high-dimensional semantic space multi-dimensional feature vector representing each image patch. All 3969 image patch multi-dimensional feature vectors constitute a multi-dimensional feature vector set. In a specific implementation, the multi-level feature perception network calculates the semantic distance between any two multi-dimensional feature vectors in the multi-dimensional feature vector set. The semantic distance reflects the difference in semantic content between the two image patches. Calculated using the following formula:

[0028] in: This represents the semantic distance between the multidimensional feature vectors of the i-th image patch and the j-th image patch in the multidimensional feature vector set. and These represent the 512-dimensional multidimensional feature vectors corresponding to the i-th and j-th image patches, respectively. (Operator) Represents the dot product of vectors. This represents the Euclidean norm of a vector. In practice, based on the calculated semantic distance between all image patch pairs, a global context association matrix is ​​constructed to reflect the global dependencies among all regions of the standard illumination image data of the entire printed circuit board. The global context association matrix is ​​a 3969x3969 real symmetric matrix, and the element value in the i-th row and j-th column is the semantic distance. semantic distance The smaller the value, the more semantically similar the two image patches are. When parsing the global context association matrix, the candidate recognition module scans the clusters of non-zero elements within the matrix. In practice, these clusters are represented by the coordinates of rows and columns whose values ​​are significantly lower than those of the surrounding areas. The module locates the coordinates of these feature abrupt changes. The feature density values ​​within a preset neighborhood are then calculated for each coordinate location. The preset neighborhood is defined as a circular region centered on the image patch corresponding to that coordinate location with a radius of three image patches. By calculating the semantic distance of all neighborhoods It is obtained by taking the reciprocal of the average value. In specific implementation, regions with feature density values ​​exceeding a dynamic threshold are selected. The dynamic threshold... The calculation is based on the distribution of all feature density values ​​in the current global context association matrix, and the formula is as follows:

[0029] in: It is the mean of all feature density values. This is its standard deviation. As can be understood, see Table 1, which shows the feature density values ​​and selection results of some image patches parsed from the global context association matrix.

[0030] Table 1: Feature Density Values ​​and Screening Results of Selected Image Patches

[0031] In some embodiments, based on the screening results, the system crops out image patch regions with feature density values ​​exceeding a dynamic threshold from the original printed circuit board standard illumination image data and marks them as high-interest candidate regions. Optionally, in terms of data comparison, for a printed circuit board standard illumination image data containing normal solder joints and cold solder defects, the average feature density value of image patches in the normal solder joint region is 0.10, while the average feature density value of image patches in the cold solder defect region is 0.15. The dynamic threshold is calculated to be 0.12, thus successfully screening the image patch corresponding to the cold solder defect region as a high-interest candidate region. In specific implementations, the size of the preset neighborhood and the coefficient of the dynamic threshold can be adjusted according to the complexity of different industrial scenarios. It can be understood that the construction of the global context association matrix depends on the calculation of the semantic distance between multi-dimensional feature vectors, and the calculation method of semantic distance determines the measurement standard of association. Optionally, in another specific implementation, for textile surface inspection, the image patch size can be set to 16x16 pixels to accommodate more subtle texture changes, and the feature density value calculation neighborhood is correspondingly adjusted to a 5x5 image patch region.

[0032] See Figure 3In the PCB inspection scenario of an attention-based industrial complex scene small target defect detection system, the optimized detection path planning for defect localization visually demonstrates the system's improved detection efficiency and accuracy by comparing the standard and optimized detection paths. Specifically, the PCB inspection workbench is a standard inspection area of ​​2048×2048 pixels, corresponding to the 18×15 visualization work area in the figure. The standard detection path is a traditional pipeline-style line-by-line scanning path, represented by solid lines. Its path planning follows a fixed robotic arm motion logic, starting from the lower left corner of the workbench and traversing the entire area horizontally and vertically. This path lacks specificity, resulting in a large number of invalid detection trips and low detection efficiency. The optimized detection path, represented by dashed lines, is an intelligent path generated by the system after iterative optimization through a multi-level feature perception network. Its path planning is entirely based on the defect target points (cross-shaped markers) identified by the system, planning the optimal trajectory only between high-attention candidate areas and areas with high defect probability, significantly shortening the detection trip. In its implementation, the system generates optimized defect location coordinates through a positioning optimization module. Combined with a path planning algorithm, it minimizes the total length of the detection path while ensuring complete coverage of all defect target points. As shown in the figure, the optimized path directly connects three PCB defect target points. Compared to the standard path's full-area traversal, this effectively reduces the robotic arm's ineffective movements and improves detection efficiency. The core logic of this path planning relies on the system's multi-level feature perception network: it identifies high-interest candidate areas through a global context association matrix, constructs a defect probability distribution heatmap, iterates and updates the network weights to correct the defect location coordinates, and finally generates the optimal detection path based on the optimized defect coordinates. This achieves a technological upgrade from "blind full-area inspection" to "targeted defect detection," adapting to the high-speed inspection needs of industrial production lines.

[0033] In one embodiment of the present invention, when extracting local texture gradient information of high-interest candidate regions, multi-directional gradient operator operations are performed on each high-interest candidate region to obtain the gradient magnitude in the horizontal direction and the gradient magnitude in the vertical direction of the image, respectively. The obtained horizontal and vertical gradient magnitudes are fused and a comprehensive texture complexity index that can comprehensively reflect the texture complexity is calculated using a specific formula. This comprehensive texture complexity index is mapped back to each corresponding pixel position of the high-interest candidate region to generate a defect probability distribution heatmap that represents the probability of a defect at that position. When outputting the preliminary defect judgment result, the judgment feedback module retrieves various historical defect images stored in the preset defect sample library and extracts the baseline heatmap features corresponding to these historical defect images. The geometric shape parameters and pixel area parameters of the high-probability regions in the current defect probability distribution heatmap are calculated, and the matching degree of these parameters is calculated with the baseline heatmap features in the preset defect sample library. Based on the matching score, the system classifies the current workpiece to be inspected as a suspected defective part or a normal part, and the classification result is recorded and output as the preliminary defect judgment result.

[0034] In practical implementation, the attention-based industrial complex scene small target defect detection system implements corresponding steps of the candidate identification module and the decision feedback module in the solar cell microcrack (EL) detection scenario. Specifically, the candidate identification module receives multiple high-interest candidate regions (HORs) identified from a standard illuminance image of a solar cell from the feature perception module. Each HOR corresponds to a rectangular region in the image that may contain microcracks. When extracting the local texture gradient information of the HORs, a multi-directional gradient operator operation is performed on each HOR. The multi-directional gradient operator operation uses the Sobel operator to calculate the gradient magnitude in the horizontal and vertical directions of the image, respectively. In practical implementation, for a HOR with a size of 32x32 pixels, the horizontal Sobel operator... Sobel operator in the vertical direction The horizontal gradient image is calculated by convolving it with the image of the region. Gradient image in the vertical direction Horizontal gradient magnitude For pixels exist The absolute value of the gradient in the vertical direction. For pixels exist The absolute value in [the equation]. In some embodiments, the horizontal gradient magnitude is fused. gradient magnitude in the vertical direction The comprehensive texture complexity index is calculated using a comprehensive calculation function. The formula for calculating the overall texture complexity index is as follows:

[0035] in: The comprehensive texture complexity index represents the p-th high-attention candidate region. This represents the set of all pixels contained in the p-th high-attention candidate region. Representative set The total number of pixels in the middle. Representing pixels The horizontal gradient magnitude, Representing pixels The vertical gradient magnitude, This represents the natural logarithm function. In practice, the comprehensive texture complexity index of each high-interest candidate region is calculated. Mapping back to the corresponding pixel position of the high-interest candidate region generates an image with the same data size as the original standard illuminance image of the solar cell, but with each pixel position assigned the value of its corresponding high-interest candidate region. The heatmap shows the probability distribution of defects, with higher values ​​indicating a greater likelihood of defects. Essentially, the heatmap visually represents the probability of hidden cracks at different locations.

[0036] When outputting the preliminary defect judgment result, the judgment feedback module retrieves historical images of microcrack defects in solar cells stored in a preset defect sample library, and extracts the baseline heatmap features of the historical defect images. These baseline heatmap features include the shape and area parameters of high-probability regions in the corresponding heatmaps of the historical defect images. The shape parameter is the aspect ratio of the rectangle fitted to the high-probability region, and the area parameter is the number of pixels occupied by the high-probability region. In specific implementation, the shape and area parameters of each high-probability region in the defect probability distribution heatmap of the solar cell to be inspected are calculated. A high-probability region is defined as one whose value in the defect probability distribution heatmap is greater than a global threshold. For each high-probability region in the connected regions, its shape and area parameters are matched with the baseline heatmap features of all historical defect images in the preset defect sample library, and a matching score is obtained. The result is obtained by calculating the reciprocal of the Euclidean distance between the parameters of the region to be tested and the parameters of historical samples. In some embodiments, the solar cells to be tested are classified according to their matching scores, and a matching score threshold is set. If the matching score corresponding to any high-attention candidate region Exceeding the matching score threshold If the condition is not met, the current solar cell is classified as a suspected defective component; otherwise, it is classified as a normal component. The classification result is recorded as the preliminary defect determination result. See Table 2 for the matching calculations of several high-interest candidate regions.

[0037] Table 2: Matching Score and Judgment Result of High-Attention Candidate Region

[0038] In practice, a pre-set defect sample database is continuously updated, storing newly confirmed defective component images and their defect probability distribution heatmap features. Optionally, in terms of data comparison, in the defect probability distribution heatmap of a normal solar cell, the maximum matching score for the high-probability area is 0.09, which is lower than the matching score threshold of 0.15, and it is judged as a normal component. A solar cell containing a real microcrack, however, has a corresponding crack area aspect ratio of 8.7 in its defect probability distribution heatmap, with a matching score of 0.22, which is higher than the matching score threshold, and is judged as a suspected defective component. The matching score threshold can be understood as... Adjustments can be made according to product acceptance standards. Optionally, in integrated circuit wafer inspection scenarios, the extraction of local texture gradient information can employ different gradient operator combinations for layers of different materials, while the generation principle of the defect probability distribution heatmap remains the same. In specific implementations, the calculation of shape and area parameters is based on the binarized high-probability connected regions.

[0039] See Figure 4In the validation of the solar cell microcrack (EL) detection scenario of the attention-based industrial complex scene small target defect detection system, the figure intuitively presents the evolution of the system's core performance indicators over six consecutive inspection batches. The dotted curve represents the accuracy rate of normal parts, and the box curve represents the detection rate of defective parts. From the performance trend, both core indicators show a continuous and stable improvement, fully validating the effectiveness of the closed-loop iterative optimization mechanism of the system's judgment feedback module: the system feeds back the preliminary defect judgment results of each batch to a multi-level feature perception network, triggering iterative updates of the network's internal weights, enabling the model to continuously optimize its feature extraction capabilities and defect recognition logic with continuous input of industrial field data. Specifically, the accuracy rate of normal parts gradually increased from 0.920 in the first batch to 0.980 in the sixth batch, indicating a continuous reduction in the system's misjudgment rate of defect-free solar cells, effectively reducing over-inspection losses during production; the detection rate of defective parts increased from 0.850 in the first batch to 0.960 in the sixth batch, achieving a significant reduction in the missed detection rate of small target microcrack defects, ensuring the reliability of product quality upon delivery. The simultaneous improvement of the two indicators verifies that, under complex industrial lighting conditions, the system, through the synergistic effect of multi-view image registration, adaptive lighting compensation, multi-level feature perception network and closed-loop feedback optimization, can continuously enhance the ability to identify micro-crack defects while ensuring normal product pass rate, thus meeting the industrial-grade requirements of solar cell production lines for high accuracy and high detection rate of defects.

[0040] In one embodiment of the present invention, the judgment feedback module feeds back the preliminary defect judgment result to the multi-level feature perception network. The network analyzes the missed areas that were misjudged as normal parts in the result and extracts the original feature data of these missed areas from the original data. The difference loss value between the original feature data of the missed areas and the features output by the current multi-level feature perception network for that area is calculated. Based on the calculated difference loss value, the gradient is backpropagated to each hidden layer of the multi-level feature perception network through the backpropagation algorithm, adjusting the weight parameters of the convolution kernels in the network to complete one internal weight iteration update. After completing the internal weight iteration update, the positioning optimization module loads the updated multi-level feature perception network and inputs the same standard illumination image data again. The network repeatedly executes the feature extraction and correlation matrix construction process to obtain a new global context correlation matrix after weight correction. The coordinate point information with the maximum response value is extracted from the corrected global context correlation matrix, and the pixel coordinates are converted into optimized defect positioning coordinates in the actual industrial coordinate system through a pre-calibrated coordinate transformation relationship. The system also continuously collects real-time environmental dust concentration data from the industrial site and simultaneously monitors the cleanliness status of the vision sensor lens. When the real-time environmental dust concentration exceeds the preset warning line, or the lens cleanliness is below the set standard value, the system automatically adjusts the filter intensity coefficient of the adaptive brightness compensation template in the preprocessing module. The adjusted filter intensity coefficient is then applied to the real-time processing of the original scene image set to regenerate standard illumination image data adapted to the environmental conditions at that time.

[0041] In practical implementation, the attention-based industrial complex scene small target defect detection system implements a judgment feedback module, a positioning optimization module, and environmental monitoring functions in a chip packaging inspection scenario. Micrometer-level solder joint defects exist on the chip packaging surface. In this implementation, the judgment feedback module feeds back the preliminary defect judgment results from the standard illumination image data of the chip packaging to a multi-level feature perception network. The preliminary defect judgment results indicate three suspected defect areas in a single chip packaging image, but manual re-inspection confirms a fourth, undetected, tiny defect area. The multi-level feature perception network analyzes the undetected area that was misjudged as a normal component in the preliminary defect judgment results and extracts the original feature data of the undetected area. The original feature data is the original RGB pixel value matrix of the corresponding pixel block in the standard illumination image data for the undetected area. In some embodiments, the difference loss value between the original feature data of the undetected area and the output feature of the corresponding area by the current multi-level feature perception network is calculated. The output feature of the multi-level feature perception network for the corresponding area is a 512-dimensional multi-dimensional feature vector. The original feature data of the missed detection region is mapped into feature vectors of the same dimension by an encoder network. Difference loss value Calculated using the following formula:

[0042] in: Represents the difference loss value. The original feature data representing the missed detection region is encoded to obtain a 512-dimensional feature vector. This represents the 512-dimensional multidimensional feature vector output by the current multi-level feature perception network for the missed detection region. This represents the L2 norm of the vector. Based on the calculated difference loss value... The gradient is backpropagated to each hidden layer of the multi-level feature perception network using the backpropagation algorithm, adjusting the weight parameters of the convolutional kernels in the hidden layers to complete one internal weight update. In practice, data comparison shows that for the same missed solder joint defect, the difference in loss value before the weight update is... The difference loss value for the same defect area is 0.15 after three feedback iterations. Reduced to 0.08.

[0043] After completing the iterative update of the network's internal weights, the localization optimization module loads the updated multi-level feature perception network and inputs the same standard illumination image data of the chip package again. The updated multi-level feature perception network repeatedly executes the feature extraction and correlation matrix construction process, which includes segmenting the image into image patches, extracting edge contour features and color distribution features, mapping to a high-dimensional semantic space to generate a multi-dimensional feature vector set, calculating semantic distance, and constructing a global context correlation matrix to obtain the corrected global context correlation matrix. In specific implementation, the coordinate information of the maximum response point is extracted from the corrected global context correlation matrix. The maximum response point corresponds to the row and column index of the element with the smallest semantic distance value in the corrected global context correlation matrix. This index corresponds to the image patch sequence number. Through the mapping relationship between the index and the image patch position, the pixel coordinates of the maximum response point in the image can be calculated. In some embodiments, pixel coordinates The optimized defect location coordinates are converted to the industrial coordinate system. This conversion is accomplished using a pre-calibrated camera parameter matrix. The following formula can be used to obtain:

[0044] in: It is a three-dimensional spatial coordinate system in the industrial coordinate system. These are the pixel coordinates in the image. It is the inverse matrix of the intrinsic parameter matrix obtained in advance through camera calibration. It can be understood that the optimized defect location coordinates will be directly used to guide the robotic arm to perform defect re-inspection or marking. Optionally, in terms of data comparison, for the same missed weld point, the error between the defect location coordinates generated before the weight update and the actual defect center is 5 pixels, and the error between the optimized defect location coordinates recalculated after the weight iteration update and the actual defect center is reduced to 1 pixel.

[0045] The system also continuously collects real-time environmental dust concentration data from the industrial site and simultaneously monitors the cleanliness of the vision sensor lens. Real-time environmental dust concentration data is collected in milligrams per cubic meter (mg / m³) via a laser dust sensor, and lens cleanliness is monitored by analyzing the sharpness score of a reference image on the lens protective glass. In specific implementations, when the real-time environmental dust concentration data exceeds a preset warning line or the lens cleanliness falls below a preset standard value, the system automatically adjusts the filter strength coefficient of the adaptive brightness compensation template in the preprocessing module. The filter strength coefficient refers to the standard deviation parameter of the Gaussian smoothing filter used in generating the adaptive brightness compensation template. In some embodiments, the default value of the filter strength coefficient of the adaptive brightness compensation template is 1.5. When the real-time environmental dust concentration data exceeds the warning line, the filter strength coefficient is adjusted to 2.0 through linear interpolation based on the excess ratio. When the lens cleanliness falls below the standard value, the filter strength coefficient is adjusted to 1.8 through linear interpolation based on the reduction ratio of the score. It is understandable that the adjusted filter intensity coefficient is applied to the processing of the original scene image set. When constructing the adaptive brightness compensation template, the new filter intensity coefficient is used for Gaussian smoothing convolution to regenerate a new version of standard illumination image data adapted to the dust concentration or lens cleanliness conditions at that time. Optionally, in the chip packaging and testing production line, if an alarm record shows that the real-time environmental dust concentration data reaches 120% of the warning line, the system adjusts the filter intensity coefficient from 1.5 to 1.8. After reprocessing the image, artifacts caused by dust scattering light are more effectively suppressed in the standard illumination image data.

[0046] See Figure 5In the defect localization accuracy optimization stage of an attention-based industrial complex scene small target defect detection system, the quantitative comparison of localization error is presented intuitively by box plots, showing the performance difference before and after weight updates. The horizontal axis corresponds to the experimental conditions "before weight update" and "after three iterations of optimization", and the vertical axis represents the pixel error value of defect localization. The core optimization logic is reflected in the weight iteration of the multi-level feature perception network and the reconstruction of the global context association matrix. Before weight updates, the localization error of chip package solder joint defects is distributed in the range of 3 to 7 pixels, and the median of the box plot is about 4.8 pixels, reflecting that the network's feature perception ability for micron-sized small target defects is insufficient in the initial state, and it is easy to miss detection and localization deviation. After three weight iterations of optimization based on the difference loss of missed detection areas, the localization error is significantly compressed to the range of 0 to 1.5 pixels, the median of the box plot is reduced to about 1 pixel, and outliers are effectively suppressed. This result confirms that after adjusting the convolution kernel weights through backpropagation and reconstructing the global context association matrix, the network's accuracy in calculating the feature gradient and semantic distance of high-attention candidate areas is greatly improved. From a quantitative perspective, the positioning error of the same missed weld point before and after optimization was reduced from 5 pixels to 1 pixel, and the difference loss value also converged from 0.15 to 0.08. This fully demonstrates that the system has achieved significant optimization of defect positioning accuracy through the weight iteration mechanism triggered by the judgment feedback module. At the same time, combined with the dynamic adjustment of the adaptive brightness compensation template by the industrial site environment monitoring module, the stability of positioning accuracy under complex working conditions is further guaranteed.

[0047] In one embodiment of the present invention, the attention-based industrial complex scene small target defect detection system implements an image acquisition module in the application scenario of lithium battery cell manufacturing production line. The image acquisition module arranges multiple heterogeneous vision sensors along the transmission direction of the cell winding and stacking production line. The heterogeneous vision sensors include a high-resolution line array camera and an infrared thermal imaging camera. The heterogeneous vision sensors synchronously trigger and capture the front view, side view, and top view images of the lithium battery cell during the winding, stacking, and welding processes. In a specific implementation, the system records the acquisition time of the front view, side view, and top view images and the QR code workpiece identification code laser-marked on the cell shell to form an image data packet with time sequence attributes. Spatial registration is performed on the image data packets. This operation extracts corner features of the QR code workpiece identification code from the front, side, and top views, along with four highly reflective physical markers pre-attached to the cell fixture. Based on the pixel coordinates of these physical markers in each view image, a perspective transformation matrix is ​​calculated. This matrix is ​​then used to perform geometric projection correction on each view image to eliminate geometric distortion caused by differences in shooting angles. The geometrically corrected view images are then time-series aligned according to the time-series attributes of the acquisition time. The multi-view corrected images corresponding to the same QR code workpiece identification code are then fused according to a preset spatial stitching rule to generate an original scene image set containing complete surface information of the cell. In some embodiments, the preprocessing module performs uneven illumination elimination processing on the original scene image set. This module analyzes the brightness distribution histogram of each pixel in the original scene image set and identifies non-uniform illumination areas with strong light reflection or electrolyte reflection. An adaptive brightness compensation template is constructed based on the location information of these non-uniform illumination areas. All pixels in the original scene image set are traversed, and the grayscale value of each pixel is adjusted according to the adaptive brightness compensation template, outputting standard illumination image data. The feature perception module inputs the standard illumination image data into the multi-level feature perception network. The multi-level feature perception network segments the standard illumination image data into overlapping image blocks and extracts the edge contour features and color distribution features of each image block. The edge contour features and color distribution features are mapped to a high-dimensional semantic space to generate a set of multi-dimensional feature vectors. The semantic distance between any two vectors in the set of multi-dimensional feature vectors is calculated, and a global context association matrix is ​​constructed based on the semantic distance.

[0048] In practical implementation, the candidate identification module parses the global context association matrix, scans the non-zero element clusters in the matrix and locates the coordinates of feature abrupt changes, counts the feature density values ​​in the preset neighborhood around the coordinates, and filters out regions whose feature density values ​​exceed a dynamic threshold. These filtered regions are then cropped from the standard illumination image data and labeled as high-interest candidate regions. These high-interest candidate regions correspond to potential defect locations such as misaligned cell electrodes, weld bubbles, or separator wrinkles. Local texture gradient information of the high-interest candidate regions is extracted, and multi-directional gradient operator operations are performed to obtain horizontal and vertical gradient magnitudes. The horizontal and vertical gradient magnitudes are then fused, and a comprehensive texture complexity index is calculated. This comprehensive texture complexity index is mapped back to the corresponding pixel positions of the high-interest candidate regions to generate a defect probability distribution heatmap. The judgment feedback module compares the similarity index of the defect probability distribution heatmap with a preset defect sample library. The preset defect sample library stores the baseline heatmap features of historical lithium battery defect images. It calculates the shape and area parameters of high-probability regions in the defect probability distribution heatmap and matches them with the baseline heatmap features. Based on the matching score, the cell is classified as a suspected defective component or a normal component, and a preliminary defect judgment result is output. The preliminary defect judgment result is fed back to the multi-level feature perception network. The network analyzes the missed detection areas that were misjudged as normal components and extracts the original feature data of these missed detection areas. It calculates the difference loss value between the original feature data of the missed detection areas and the current output features of the multi-level feature perception network. Based on the difference loss value, the network backpropagates to the hidden layers and adjusts the weight parameters of the convolutional kernels to complete the iterative update of the network weights. After completing the iterative update of the network's internal weights, the localization optimization module recalculates the global context association matrix. It then loads the multi-level feature perception network after the internal weight update and re-inputs standard illumination image data. The feature extraction and association matrix construction process is repeated to obtain the corrected global context association matrix. From this corrected matrix, the coordinates of the maximum response point are extracted and converted into optimized defect localization coordinates in the robotic arm coordinate system. This process can be understood to enable the detection and localization of small-target defects in lithium-ion battery cell manufacturing, including electrode alignment accuracy in the winding process and separator flatness in the stacking process.

[0049] In the scenario of inspecting gear components for automotive electronic parking brake systems, an image acquisition module is implemented. The image acquisition module is equipped with multiple heterogeneous vision sensors arranged along the transmission direction of the gear assembly line. The heterogeneous vision sensors include multiple area array cameras arranged in a ring. The heterogeneous vision sensors are simultaneously triggered to capture front view, side view, and top view images of the first-stage gear, second-stage gear, and planetary gear assembly. In specific implementation, the system records the acquisition time of the front view, side view, and top view images and the serial number workpiece identification code etched on the end face of the gear, forming an image data package with time sequence attributes. Spatial registration is performed on the image data packets. This operation extracts the character features of the serial number workpiece identification code and the physical marker points on the edge of the gear mounting hole from the front, side, and top views. Based on the pixel coordinates of the physical marker points in each view image, a perspective transformation matrix is ​​calculated between the different view images. This matrix is ​​then used to perform geometric projection correction on each view image to eliminate geometric distortion caused by differences in shooting angles. The geometrically corrected view images are then time-series aligned according to the time-series attributes of the acquisition time. The multi-view corrected images corresponding to the same serial number workpiece identification code are fused according to a preset spatial stitching rule to generate an original scene image set containing information on all tooth surfaces and end faces of the gear. In some embodiments, the preprocessing module performs uneven illumination elimination processing on the original scene image set. The preprocessing module analyzes the brightness distribution histogram of each pixel in the original scene image set and identifies non-uniform illumination areas where cutting fluid reflection or oil stains obscure the light. An adaptive brightness compensation template is constructed based on the location information of the non-uniform illumination areas. All pixels in the original scene image set are traversed, and the grayscale value of each pixel is adjusted according to the adaptive brightness compensation template to output standard illumination image data. The feature perception module inputs the standard illumination image data into the multi-level feature perception network. The multi-level feature perception network segments the standard illumination image data into overlapping image blocks and extracts the edge contour features and color distribution features of each image block. The edge contour features and color distribution features are mapped to a high-dimensional semantic space to generate a set of multi-dimensional feature vectors. The semantic distance between any two vectors in the set of multi-dimensional feature vectors is calculated, and a global context association matrix is ​​constructed based on the semantic distance.

[0050] In some embodiments, the candidate recognition module parses the global context association matrix, scans the non-zero element clusters in the global context association matrix and locates the coordinates of feature abrupt changes, counts the feature density values ​​in the preset neighborhood around the coordinates, and filters out regions whose feature density values ​​exceed a dynamic threshold. The filtered regions are cropped from the standard illumination image data and marked as high-interest candidate regions. These high-interest candidate regions correspond to potential defect locations such as rust on gear teeth, foreign objects on gear teeth, excess material on gear teeth, flash in inner holes, gear burrs, or gear impact damage. Local texture gradient information of the high-interest candidate regions is extracted, and multi-directional gradient operator operations are performed on them to obtain horizontal and vertical gradient magnitudes. The horizontal and vertical gradient magnitudes are fused, and a comprehensive texture complexity index is calculated. This comprehensive texture complexity index is mapped back to the corresponding pixel positions of the high-interest candidate regions to generate a defect probability distribution heatmap. The judgment feedback module compares the similarity index of the defect probability distribution heatmap with a preset defect sample library. The preset defect sample library stores the baseline heatmap features of historical gear defect images. It calculates the shape and area parameters of high-probability regions in the defect probability distribution heatmap and matches them with the baseline heatmap features. Based on the matching score, the gear assembly is classified as a suspected defective part or a normal part, and a preliminary defect judgment result is output. The preliminary defect judgment result is fed back to the multi-level feature perception network. The network analyzes the missed areas that were misjudged as normal parts and extracts the original feature data of these missed areas. It calculates the difference loss value between the original feature data of the missed areas and the current output features of the multi-level feature perception network. Based on the difference loss value, the network backpropagates to the hidden layers and adjusts the weight parameters of the convolutional kernels to complete the iterative update of the network weights. After completing the iterative update of the network's internal weights, the localization optimization module recalculates the global context association matrix, loads the multi-level feature perception network after the internal weight update, and re-inputs the standard illumination image data. It then repeats the feature extraction and association matrix construction process to obtain the corrected global context association matrix. From this corrected matrix, it extracts the coordinates of the maximum response point and converts them into optimized defect localization coordinates in the robot coordinate system. This means that in the automotive gear assembly inspection scenario, defects such as tooth decay in the first-stage gear and insufficient material in the reinforcing ribs of the second-stage gear can be detected and located using the above process. Optionally, the system also collects real-time environmental dust concentration data from the gear assembly station and monitors the lens cleanliness of the vision sensor. When the real-time environmental dust concentration exceeds the warning line or the lens cleanliness is below the standard value, the system automatically adjusts the filter intensity coefficient of the adaptive brightness compensation template and applies the adjusted coefficient to the processing of the original scene image set to regenerate the standard illumination image data.Optionally, the acquisition time and workpiece identification code information attached to each image in the gear assembly image data package are stored in a structured database for quality traceability and analysis of production batches.

[0051] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An attention mechanism-based industrial complex scene small target defect detection system, characterized in that, include: The image acquisition module is equipped with a panoramic vision acquisition device for the industrial site to acquire the original scene image set of the workpiece to be inspected; The preprocessing module performs illumination unevenness elimination processing on the original scene image set to generate standard illumination image data; The feature perception module inputs the standard illumination image data into a multi-level feature perception network and calculates the global context association matrix, including: The standard illumination image data is divided into several overlapping image blocks, and the edge contour features and color distribution features of each image block are extracted respectively; The edge contour features and color distribution features are mapped to a high-dimensional semantic space to generate a set of multi-dimensional feature vectors. Calculate the semantic distance between any two vectors in the multidimensional feature vector set, and construct a global context association matrix reflecting global dependencies based on the semantic distance; The candidate identification module parses the global context association matrix, identifies potential small target defect regions, marks them as high-interest candidate regions, extracts the local texture gradient information of the high-interest candidate regions, and constructs a defect probability distribution heatmap. The judgment feedback module compares the similarity index of the defect probability distribution heatmap with the preset defect sample library and outputs the preliminary defect judgment result. The preliminary defect determination result is fed back to the multi-level feature perception network, triggering iterative updates of the network's internal weights. The localization optimization module, after completing the iterative update of the internal weights of the network, recalculates the global context association matrix and generates optimized defect localization coordinates.

2. The attention mechanism based industrial complex scene small target defect detection system of claim 1, wherein, The configuration of the industrial site panoramic vision acquisition device, which acquires the original scene image set of the workpiece to be inspected, includes: Multiple heterogeneous vision sensors are arranged along the conveying direction of the industrial production line to simultaneously capture the front view, side view, and top view of the workpiece to be inspected. Record the acquisition time and workpiece identification code of the front view, side view and top view images to form an image data packet with time sequence attributes; Spatial registration is performed on the image data packets to eliminate geometric distortions between images from different viewpoints and integrate them to generate an original scene image set of the workpiece to be inspected, specifically including: Extract the workpiece identification code feature points from the front view image, side view image and top view image, and locate at least four non-collinear physical marker points in each view image; Based on the pixel coordinates of the physical markers in the images from different viewpoints, the perspective transformation matrix between the images from different viewpoints is calculated, and the perspective transformation matrix is ​​used to perform geometric projection correction on the images from different viewpoints to eliminate geometric distortion caused by differences in shooting angles. Based on the time sequence attributes of the acquisition time, the images of each view after geometric projection correction are time-series aligned, and the multi-view corrected images corresponding to the same workpiece identification code are fused according to the preset spatial stitching rules to generate an original scene image set of the workpiece to be inspected containing complete surface information.

3. The attention mechanism based industrial complex scene small target defect detection system of claim 2, wherein, Performing illumination unevenness elimination processing on the original scene image set to generate standard illumination image data includes: Analyze the brightness distribution histogram of each pixel in the original scene image set to identify non-uniform lighting areas with strong light reflection or shadow occlusion; Based on the location information of the non-uniform illumination area, an adaptive brightness compensation template is constructed; Traverse all pixels of the original scene image set, adjust the grayscale value of each pixel according to the adaptive brightness compensation template, and output standard illumination image data. The step of constructing an adaptive brightness compensation template based on the location information of the non-uniform illumination region includes: A morphological dilation operation is performed on the non-uniform illumination region to expand the region boundary to cover the transition illumination band, generating a composite illumination mask that includes a core illumination anomaly region and a transition region. Within the composite illumination mask, a 5×5 pixel local neighborhood window is divided with each pixel as the center. The mean and standard deviation of the pixel grayscale within the window are calculated as the local illumination statistics of that pixel. Based on the local illumination statistics, a bicubic interpolation algorithm is used to fill in the missing brightness values ​​in the non-uniform illumination area to generate an initial brightness compensation matrix. A Gaussian smoothing filter is introduced to convolve the initial brightness compensation matrix to filter out high-frequency noise interference, resulting in a smoothed adaptive brightness compensation template.

4. The attention mechanism based industrial complex scene small target defect detection system of claim 3, wherein, Parsing the global context correlation matrix identifies potential small target defect regions, which are then marked as high-interest candidate regions, including: Scan the non-zero element clusters in the global context association matrix to locate the coordinates of feature mutations; Statistically analyze the feature density values ​​within a preset neighborhood around the coordinate position, and filter out regions where the feature density values ​​exceed a dynamic threshold. The selected regions are cropped from the standard illumination image data and marked as high-interest candidate regions.

5. The attention mechanism based industrial complex scene small target defect detection system of claim 4, wherein, Extracting the local texture gradient information of the high-interest candidate region and constructing a defect probability distribution heatmap includes: Perform multi-directional gradient operator operations on the high-interest candidate region to obtain the horizontal gradient magnitude and the vertical gradient magnitude; The horizontal and vertical gradient magnitudes are combined to calculate the overall texture complexity index. The comprehensive texture complexity index is mapped back to the corresponding pixel position of the high-attention candidate region to generate a heatmap of defect probability distribution with probability values.

6. The attention mechanism based industrial complex scene small target defect detection system of claim 5, wherein, By comparing the similarity index between the defect probability distribution heatmap and the preset defect sample library, the preliminary defect judgment result is output, including: Retrieve historical defect images stored in the preset defect sample library and extract the baseline heatmap features of the historical defect images; Calculate the shape and area parameters of high-probability regions in the defect probability distribution heatmap and match them with the features of the baseline heatmap; Based on the matching score, the workpiece to be inspected is classified as a suspected defective part or a normal part, and the classification result is recorded as a preliminary defect judgment result.

7. The industrial complex scene small target defect detection system based on attention mechanism as described in claim 6, characterized in that, Feeding the preliminary defect determination result back to the multi-level feature perception network and triggering iterative updates of the network's internal weights includes: Analyze the missed areas that were misjudged as normal parts in the preliminary defect determination results, and extract the original feature data of the missed areas; Calculate the difference loss value between the original feature data of the missed detection region and the output features of the current multi-level feature perception network; The difference loss value is backpropagated to the hidden layer of the multi-level feature perception network to adjust the weight parameters of the convolution kernel, thus completing one internal weight iteration update of the network.

8. The industrial complex scene small target defect detection system based on attention mechanism as described in claim 7, characterized in that, After completing the iterative update of the network's internal weights, the global context association matrix is ​​recalculated to generate optimized defect location coordinates, including: After loading the multi-level feature perception network with iterative weight updates within the network, the standard illumination image data is input again. Repeat the feature extraction and association matrix construction process to obtain the corrected global context association matrix; The coordinate information of the maximum response point is extracted from the modified global context association matrix and converted into optimized defect location coordinates in the industrial coordinate system.

9. The industrial complex scene small target defect detection system based on attention mechanism as described in claim 8, characterized in that, The system also includes: Collect real-time environmental dust concentration data at industrial sites and monitor the cleanliness of the vision sensor lens; When the real-time environmental dust concentration data exceeds the warning line or the lens cleanliness status is lower than the standard value, the filter intensity coefficient of the adaptive brightness compensation template is automatically adjusted. The adjusted filter intensity coefficients are applied to the processing of the original scene image set to regenerate standard illumination image data.

Images