Method and system for intelligently detecting assembly state of product parts

Abstract

The invention relates to the technical field of image recognition, in particular to a method and system for intelligently detecting the assembly state of product parts. Comprising the following steps: driving an industrial camera to perform multi-point image acquisition on a product through a servo module; performing image registration on the local image and the corresponding template image to obtain a registration transformation matrix; calculating the offset of part assembly based on the matrix, comparing the offset with a threshold value, and judging whether the assembly is qualified or not; if the detection result is qualified, generating a key detection area based on offset and contour information, and mapping the key detection area to a local image through inverse transformation; constructing an attention grey-scale map, and guiding image segmentation to obtain a plurality of image segments; and inputting the image segments into a convolutional neural network model to carry out assembly anomaly classification detection, and fusing local detection results to output final judgment. The system is high in detection accuracy and speed, and is suitable for assembly quality detection in a complex scene.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and system for intelligently detecting the assembly status of product parts. Background Art

[0002] Currently, the new energy electric vehicle battery box manufacturing market often experiences assembly errors and omissions during post-production assembly. Existing customers use on-site manual visual inspections to check for assembly errors and omissions before shipping finished products. This inspection method requires high job skills and a long period of pre-job training. Because components such as rivet nuts, rivet bolts, machined holes, brackets, stickers, and glue are relatively small, this inspection method relies on workers to check specific parts one by one. This method is inefficient and can easily cause visual fatigue during long inspections, leading to missed inspections. This limits daily production efficiency and prevents significant increases in enterprise capacity.

[0003] In existing technologies, some new energy battery box manufacturers have attempted to introduce automated inspection methods based on image recognition algorithms to replace manual visual inspection. However, existing image recognition solutions generally suffer from the following drawbacks: First, traditional algorithms mostly rely on fixed-angle shooting and static template comparison, making them difficult to adapt to complex assembly environments with subtle part position shifts, occlusions, and lighting changes, resulting in insufficient detection accuracy. Second, some systems lack effective regional attention mechanisms, often uniformly processing the entire image and failing to focus on key assembly areas.

[0004] Therefore, a method and system for intelligently detecting the assembly status of product parts are proposed. Summary of the Invention

[0005] This invention provides a method and system for intelligently detecting the assembly status of product parts. Based on the registration transformation between a template image and a local image, the system calculates the part assembly offset and generates an asymmetric tolerance band, thereby constructing a critical inspection area. Furthermore, an image segmentation strategy guided by an attention grayscale image is employed to extract highly relevant image segments, and a convolutional neural network model is used to classify and determine the assembly status. This method achieves high-precision automatic detection of the assembly status of complex parts and is suitable for intelligent quality inspection scenarios for products such as battery packs for new energy vehicles.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for intelligently detecting the assembly status of product parts, comprising:

[0008] S100: The servo module drives the industrial camera to move to the shooting point, captures images of a local area of ​​the battery box, and obtains a local image;

[0009] S200: performing image registration on the local image and the corresponding template image to obtain a registration transformation matrix from the template image to the local image;

[0010] S300: Based on the registration transformation matrix, the offset of the part assembly is calculated and compared with a preset offset threshold. If the offset is greater than the offset threshold, the part assembly state is determined to be unqualified. Otherwise, the key detection area of ​​the template image is defined based on the offset, and reverse mapping is performed in combination with the inverse transformation of the registration transformation matrix to obtain the target detection area in the local image.

[0011] S400: The target detection area is used as the attention-guiding area, and other areas are used as non-attention-guiding areas. An attention grayscale map is constructed, and the attention grayscale map is channel-joined with the local image.

[0012] S500: performing image segmentation on the spliced ​​local image, dividing it into image segments and inputting them into the trained convolutional neural network model to perform classification detection of assembly anomalies;

[0013] S600: Based on the output of the convolutional neural network, the detection results of each image segment are integrated to generate the assembly detection result of the local area and store it to the disk;

[0014] S700: Repeat steps S100 to S600 until all areas of the battery box to be inspected are traversed, and generate a battery box assembly result through Boolean operations.

[0015] Furthermore, the step of obtaining a registration transformation matrix from the template image to the local image includes:

[0016] Extract salient feature points from the template image and the local image;

[0017] Calculate matching pairs based on the feature points and use the RANSAC algorithm to eliminate abnormal matching pairs;

[0018] The registration transformation matrix from the template image to the local image is obtained by fitting the affine transformation through the matching point set.

[0019] Furthermore, the steps of calculating the offset of the part assembly include:

[0020] Extract the centroid coordinates of the preset part area in the template image, and extract the actual centroid coordinates of the corresponding part in the local image;

[0021] The centroid coordinates of the template are mapped to the local image coordinate system through the registration transformation matrix to obtain the theoretical centroid coordinates;

[0022] Calculate the Euclidean distance and direction angle deviation between the theoretical centroid coordinates and the actual centroid coordinates.

[0023] Furthermore, the step of obtaining the target detection area in the local image includes:

[0024] Extract the part outline in the template image based on the edge detection algorithm;

[0025] The calculated offset is asymmetrically expanded to generate a tolerance zone, and the key inspection area is generated in combination with the part contour;

[0026] The key detection area is mapped to the local image coordinate system through the inverse transformation of the registration transformation matrix to obtain the corresponding target detection area.

[0027] Furthermore, the operation of constructing the attention grayscale map is:

[0028] The pixel values ​​in the target detection area are set to a preset high value, and the pixel values ​​in the remaining non-detection areas are set to a preset low value. The grayscale image is smoothed by Gaussian filtering to generate an attention grayscale image.

[0029] Furthermore, the step of performing image segmentation on the spliced ​​local image includes:

[0030] Binarize the attention grayscale image based on the preset grayscale threshold and extract the connected domain contour of the key detection area;

[0031] Calculate the minimum bounding box for each connected area as the candidate area of ​​the image segment;

[0032] The stitched local image is cropped using the candidate regions to obtain image segments.

[0033] Furthermore, the convolutional neural network model is specifically ResNet, and the attention module used is the CBAM module.

[0034] Furthermore, the steps of generating the battery box assembly result through Boolean operation include:

[0035] Obtain the assembly test results of each local area. Each test result is a binary state, indicating whether the assembly is qualified or unqualified;

[0036] A logical "OR" operation is performed on the inspection results of all local areas. If any local area is judged to be unqualified, the overall assembly status is judged to be unqualified.

[0037] The present invention also provides a system for intelligently detecting the assembly status of product parts, comprising:

[0038] Image acquisition module, driven by the servo module to move the industrial camera to the shooting point, captures the image of the local area of ​​the battery box, and obtains a local image;

[0039] The template matching module is used to perform image registration between the local image and the corresponding template image to obtain a registration transformation matrix from the template image to the local image;

[0040] The offset determination module is used to calculate the offset of the part assembly based on the registration transformation matrix and compare it with the preset offset threshold. If the offset is greater than the offset threshold, the part assembly state is determined to be unqualified. Otherwise, the key detection area of ​​the template image is defined based on the offset, and reverse mapping is performed in combination with the inverse transformation of the registration transformation matrix to obtain the target detection area in the local image.

[0041] The channel augmentation module is used to construct an attention grayscale map with the target detection area as the attention-guiding area and other areas as the non-attention-guiding areas, and then perform channel splicing on the attention grayscale map and the local image;

[0042] The anomaly detection module performs image segmentation on the spliced ​​local image, divides it into image segments, and inputs them into the trained convolutional neural network model to perform classification detection of assembly anomalies;

[0043] The result analysis module is used to fuse the detection results of each image segment based on the output results of the convolutional neural network, generate the assembly detection results of the local area and store them on disk; repeatedly run the above module until all the areas to be inspected of the battery box are traversed, and generate the battery box assembly results through Boolean operations.

[0044] The beneficial effects of the present invention are:

[0045] This invention significantly improves the spatial accuracy of assembly inspection by introducing a part alignment technique based on image registration, combining offset calculation with asymmetric tolerance band construction. By extracting significant feature points between the template image and the local image and matching them using the RANSAC algorithm, a highly accurate registration transformation matrix is ​​obtained, providing a precise foundation for subsequent assembly offset calculation and critical area mapping. Furthermore, centroid coordinate comparison, combined with affine transformation, accurately assesses the degree of part offset in actual assembly, avoiding misjudgment based solely on shape overlap.

[0046] By asymmetrically expanding the part outline to construct critical detection areas, the system enhances sensitivity and discrimination to offset directions, effectively improving adaptability in complex environments. Subsequently, an attention grayscale map is constructed and smoothed, allowing the critical areas to guide image segmentation and significantly improving the relevance and effectiveness of image segments. This attention-guided image segmentation effectively focuses on areas with a high risk of assembly anomalies, enhancing the detection efficiency and accuracy of the classification neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 This is a flow chart of a method for intelligently detecting the assembly status of product parts provided by the present invention;

[0049] Figure 2 This is a structural diagram of a system for intelligently detecting the assembly status of product parts provided by the present invention;

[0050] Figure 3 This is an execution flow chart of the intelligent parts assembly detection system provided by the present invention. DETAILED DESCRIPTION

[0051] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0052] Example 1

[0053] A method for intelligently detecting the assembly status of product parts, comprising:

[0054] S100: The servo module drives the industrial camera to move to the shooting point, captures images of a local area of ​​the battery box, and obtains a local image;

[0055] Specifically, the industrial camera is mounted on the end of a servo module, which utilizes a multi-axis motion platform to control the camera's position and posture in three dimensions. A controller, based on a pre-defined inspection path planning program, moves the camera to multiple designated shooting locations. Upon reaching each location, the system triggers the camera to capture a high-definition image of the corresponding local area.

[0056] S200: performing image registration on the local image and the corresponding template image to obtain a registration transformation matrix from the template image to the local image;

[0057] Furthermore, the step of obtaining a registration transformation matrix from the template image to the local image includes:

[0058] Extract salient feature points from the template image and the local image;

[0059] Calculate matching pairs based on the feature points and use the RANSAC algorithm to eliminate abnormal matching pairs;

[0060] The registration transformation matrix from the template image to the local image is obtained by fitting the affine transformation through the matching point set.

[0061] Specifically, feature extraction is performed on the template image and the local image respectively. It is preferred to use the SIFT algorithm to extract significant feature points in the two images. Local feature algorithms such as ORB, SURF or AKAZE can also be selected according to needs to obtain image feature descriptors with rotation and scale invariance. The feature descriptors in the two images are matched using Euclidean distance or Hamming distance to form an initial set of matching point pairs, and the initial matching point pairs are screened using the RANSAC (random sampling consensus) algorithm to remove outlier matching points and retain inlier matching pairs that meet geometric constraints. Based on the retained valid matching point pairs, the least squares method is used to fit the affine transformation model to obtain the registration transformation matrix from the template image to the local image.

[0062] By extracting significant feature points and combining them with the RANSAC algorithm to eliminate abnormal matching pairs, the fitted affine transformation matrix achieves higher accuracy and anti-interference capabilities. This process enables accurate alignment between the template image and the local image, providing a foundation for subsequent offset calculations and key area mapping, ensuring accuracy and consistency throughout the entire assembly inspection process.

[0063] S300: Based on the registration transformation matrix, the offset of the part assembly is calculated and compared with a preset offset threshold. If the offset is greater than the offset threshold, the part assembly state is determined to be unqualified. Otherwise, the key detection area of ​​the template image is defined based on the offset, and reverse mapping is performed in combination with the inverse transformation of the registration transformation matrix to obtain the target detection area in the local image.

[0064] Furthermore, the steps of calculating the offset of the part assembly include:

[0065] Extract the centroid coordinates of the preset part area in the template image, and extract the actual centroid coordinates of the corresponding part in the local image;

[0066] The centroid coordinates of the template are mapped to the local image coordinate system through the registration transformation matrix to obtain the theoretical centroid coordinates;

[0067] Calculate the Euclidean distance and direction angle deviation between the theoretical centroid coordinates and the actual centroid coordinates.

[0068] Specifically, in the template image, for the part area to be detected, the image segmentation method is used to obtain the contour of the part area and calculate its two-dimensional centroid coordinates C template =(x t ,y t ), similarly, in the local image, the contour of the part area in the actual assembly state is extracted and the centroid is calculated to obtain the actual centroid coordinates C actual =(x a ,y a ). Using the registration transformation matrix obtained in the previous step, the centroid coordinates C in the template image aretemplate Mapped to the local image coordinate system, the centroid coordinate C of the theoretical assembly position is obtained theory =(x th ,y th ). The Euclidean distance and azimuth deviation (the angle between the line connecting the two points and the horizontal axis) are calculated based on the theoretical centroid coordinates and the actual centroid coordinates. The calculation of the Euclidean distance can be expressed as:

[0069]

[0070] The calculation of the azimuth deviation can be expressed as:

[0071] θ=arctan 2(y a -y th ,x a -x th );

[0072] Here, arctan represents the inverse tangent function, and the Euclidean distance and azimuth deviation are used as the offset.

[0073] By extracting the centroid coordinates of the corresponding parts in the template image and the local image, and accurately mapping the template coordinates to the local image coordinate system using a registration transformation matrix, a direct comparison between the theoretical and actual assembly positions is achieved. Furthermore, using Euclidean distance and angular deviation as measurement indicators can comprehensively reflect the assembly deviation of parts in both translation and rotation, facilitating a more accurate and intuitive assessment of the assembly status.

[0074] Furthermore, the step of obtaining the target detection area in the local image includes:

[0075] Extract the part outline in the template image based on the edge detection algorithm;

[0076] The calculated offset is asymmetrically expanded to generate a tolerance zone, and the key inspection area is generated in combination with the part contour;

[0077] The key detection area is mapped to the local image coordinate system through the inverse transformation of the registration transformation matrix to obtain the corresponding target detection area.

[0078] Specifically, for the part area to be detected in the template image, an edge detection algorithm (such as the Canny algorithm, Sobel operator or Laplacian operator) is used to extract the part contour and obtain the pixel set of the contour boundary. Where N represents the number of contour boundary points. With each contour boundary point as the center, a certain distance is non-uniformly expanded as the tolerance band, and the distance r is expanded in the offset direction. || = r0 + k1·D, vertical expansion distance r ⊥= r0 + k2·D, where r0 represents the basic tolerance, k1 and k2 represent the offset magnification coefficients. In a feasible implementation, r0 is set to 5Px, and k1 and k2 are set to 1. The expanded tolerance band is expressed as B, and the final detection area R can be expressed as in Describe the non-uniform dilation operation. The final detection area in the template image is mapped to the local image coordinate system using the inverse transformation of the registration transformation matrix to obtain the corresponding target detection area.

[0079] The edge detection algorithm accurately extracts the outline of the part in the template image and performs direction-sensitive asymmetric expansion based on the actual assembly offset to generate a tolerance band that includes potential assembly deviations, thereby constructing a more targeted key inspection area. This inspection area is then accurately mapped to the local image through the inverse mapping operation of the registration transformation matrix, effectively improving the accuracy of target area positioning.

[0080] S400: The target detection area is used as the attention-guiding area, and other areas are used as non-attention-guiding areas. An attention grayscale map is constructed, and the attention grayscale map is channel-joined with the local image.

[0081] Furthermore, the operation of constructing the attention grayscale map is:

[0082] The pixel values ​​in the target detection area are set to a preset high value, and the pixel values ​​in the remaining non-detection areas are set to a preset low value. The grayscale image is smoothed by Gaussian filtering to generate an attention grayscale image.

[0083] Specifically, a single-channel grayscale image of the same size as the local image is created, with all initial values ​​set to a low intensity value of 0. Pixel positions within the target detection area are assigned a preset high value, while pixel values ​​in the remaining non-detection areas are set to a preset low value. In one feasible implementation, the preset high value is set to 255, and the preset low value is set to 128. To prevent hard edges from affecting subsequent convolutional neural network processing, the grayscale image is smoothed using a Gaussian filter to generate an attention grayscale image.

[0084] By setting the target detection area to a high response value and the non-detection area to a low response value, and combining it with Gaussian filtering for smoothing, the attention weight of the key area can be effectively highlighted, and background interference information can be suppressed, thereby guiding subsequent image segmentation and neural network models to focus more on areas prone to assembly anomalies and reduce the misjudgment rate.

[0085] S500: performing image segmentation on the spliced ​​local image, dividing it into image segments and inputting them into the trained convolutional neural network model to perform classification detection of assembly anomalies;

[0086] Furthermore, the step of performing image segmentation on the spliced ​​local image includes:

[0087] Binarize the attention grayscale image based on the preset grayscale threshold and extract the connected domain contour of the key detection area;

[0088] Calculate the minimum bounding box for each connected area as the candidate area of ​​the image segment;

[0089] The stitched local image is cropped using the candidate regions to obtain image segments.

[0090] Specifically, the constructed attention grayscale map is binarized based on a preset grayscale threshold to generate a binary mask map. In a feasible implementation, the preset grayscale threshold is 129. The binary mask map is analyzed using the eight-neighborhood labeling algorithm to extract the connected areas consisting of all non-zero pixels, and each connected area is uniquely numbered. For each independent connected area, its minimum circumscribed rectangle (i.e., the minimum horizontal rectangular box surrounding the area) is calculated as the minimum bounding box, which is the candidate area of ​​the image segment. The spliced ​​local image is cropped using the candidate area to obtain the image segment.

[0091] Through the grayscale Figure 2 By extracting connected regions from key areas using numerical methods and calculating the minimum bounding box as candidate image fragment regions, we can achieve adaptive segmentation of key assembly areas. This method avoids the potential loss of information or redundancy associated with fixed region segmentation, helps accurately extract potential anomaly areas, and thus enhances the efficiency of anomaly detection.

[0092] Furthermore, the convolutional neural network model is specifically ResNet, and the attention module used is the CBAM module.

[0093] Specifically, ResNet consists of multiple stacked residual blocks, each of which incorporates a bottleneck design, effectively reducing the number of parameters and computational complexity. Residual blocks introduce "residual connections," enabling the network to directly learn the residual mapping rather than the original mapping. For a stacked structure, the original mapping to be learned is H(x). By introducing residual connections, these layers learn F(x) = H(x) - x, the residual portion. The network's final output is F(x) + x. This design makes deep networks easier to train and effectively addresses the vanishing gradient and degradation issues inherent in deep networks. Each residual block employs a bottleneck design, with three convolutional layers: 1×1 convolution for dimensionality reduction, 3×3 convolution for feature extraction, and finally 1×1 convolution for dimensionality restoration. Specifically, if the input feature map has 256 channels, it is first reduced to 64 channels via 1×1 convolution, then features are extracted via 3×3 convolution, and finally restored to 256 channels via 1×1 convolution. The design of first reducing the dimension and then increasing the dimension is used to greatly reduce the number of parameters and computational complexity while maintaining the network's expressive power. The attention mechanism enables the model to focus on more important information by adaptively recalibrating the weights of the feature maps. The channel attention of the CBAM module calculates the importance weight of each channel; the spatial attention focuses on the importance of the spatial position of the feature map. First, feature compression is performed through global average pooling to compress the H×W×C feature map to 1×1×C; then, feature recalibration is performed through a two-layer fully connected network to learn the dependencies between channels; finally, the learned channel weights are multiplied with the original feature map to enhance important channels and suppress unimportant channels. In a feasible implementation, ResNet adopts the ResNet-101 structure, and the CBAM module is inserted before the first residual block and after the last residual block.

[0094] The design of using ResNet convolutional neural network and introducing CBAM attention mechanism can enhance the model's perception of key areas and important channels while ensuring the depth and stability of feature extraction.

[0095] S600: Based on the output of the convolutional neural network, the detection results of each image segment are integrated to generate the assembly detection result of the local area and store it to the disk;

[0096] Specifically, each image segment is input into the trained convolutional neural network model to obtain its classification prediction result. The detection results of each image segment (such as "qualified" or "unqualified") are fused through an "or" operation, and the assembly detection results of the local area are encapsulated together with the corresponding coordinates, timestamp and other information into a JSON format data file and written to the disk.

[0097] S700: Repeat steps S100 to S600 until all areas of the battery box to be inspected are traversed, and generate a battery box assembly result through Boolean operations.

[0098] Furthermore, the steps of generating the battery box assembly result through Boolean operation include:

[0099] Obtain the assembly test results of each local area. Each test result is a binary state, indicating whether the assembly is qualified or unqualified;

[0100] A logical "OR" operation is performed on the inspection results of all local areas. If any local area is judged to be unqualified, the overall assembly status is judged to be unqualified.

[0101] Specifically, the assembly inspection results of the aforementioned local areas (such as multiple camera shooting points) are collected uniformly, and each local inspection result is set as R i ∈{0,1}, when R i =0, it means that the test result of the i-th region is qualified, otherwise it means unqualified. Through the logical "or" operation, the local test results are summarized to calculate the total assembly judgment value R total =R1∨R2∨…∨R n , when R total =1, indicating that at least one local area has failed the inspection, and the overall battery pack assembly is considered unqualified. Conversely, if all areas are qualified, the overall assembly is considered qualified. The final judgment results can be recorded in a file or database and output to the host system for device linkage or alarm prompts.

[0102] The logical "OR" operation rule is adopted to ensure that once any local area has an assembly failure, the overall assembly status can be promptly judged as unqualified, thereby improving the sensitivity and fault coverage of the detection system, effectively avoiding missed detection, and ensuring the reliability and consistency of product assembly quality.

[0103] Example 2

[0104] The present invention also proposes a system for intelligently detecting the assembly status of product parts. Figure 2 As shown, including:

[0105] Image acquisition module, driven by the servo module to move the industrial camera to the shooting point, captures the image of the local area of ​​the battery box, and obtains a local image;

[0106] The template matching module is used to perform image registration between the local image and the corresponding template image to obtain a registration transformation matrix from the template image to the local image;

[0107] The offset determination module is used to calculate the offset of the part assembly based on the registration transformation matrix and compare it with the preset offset threshold. If the offset is greater than the offset threshold, the part assembly state is determined to be unqualified. Otherwise, the key detection area of ​​the template image is defined based on the offset, and reverse mapping is performed in combination with the inverse transformation of the registration transformation matrix to obtain the target detection area in the local image.

[0108] The channel augmentation module is used to construct an attention grayscale map with the target detection area as the attention-guiding area and other areas as the non-attention-guiding areas, and then perform channel splicing on the attention grayscale map and the local image;

[0109] The anomaly detection module performs image segmentation on the spliced ​​local image, divides it into image segments, and inputs them into the trained convolutional neural network model to perform classification detection of assembly anomalies;

[0110] The result analysis module is used to fuse the detection results of each image segment based on the output results of the convolutional neural network, generate the assembly detection results of the local area and store them on disk; repeatedly run the above module until all the areas to be inspected of the battery box are traversed, and generate the battery box assembly results through Boolean operations.

[0111] Example 3

[0112] In response to the transformation of intelligent manufacturing, a battery box manufacturer adopted the intelligent parts assembly detection system described in the present invention. The system performs the following process: Figure 3 As shown in the figure, the system accurately locates key inspection areas based on a registration transformation matrix. It also incorporates asymmetric tolerance band design to construct directionally sensitive target detection areas. It also uses attention-guided image segmentation to enhance feature extraction. The system utilizes a ResNet network architecture, coupled with the CBAM attention module, to achieve high-precision classification and detection of assembly anomalies.

[0113] To verify the performance of this system, the manufacturer compared it with another competing visual inspection solution on the market, also based on ResNet+CBAM. The focus was on testing indicators such as detection accuracy, missed detection rate, false positive rate, detection speed, and lighting adaptability. The results are shown in Table 1:

[0114] Table 1 Comparison of performance indicators of the present invention and competing solutions

[0115]

[0116]

[0117] Based on the data in the table above, the system of the present invention outperforms competing solutions in key performance indicators. Specifically, the detection accuracy rate has been significantly improved, reaching 98.7% for the present invention, while the competing solution is 92.4%, an increase of 6.3 percentage points, significantly enhancing the reliability of assembly anomaly identification; the missed detection rate and false positive rate have been significantly reduced, from 4.5% and 3.1% of competing solutions to 0.8% and 1.2% respectively, effectively reducing the risk of missed detection and false positive, ensuring stable assembly quality, and increasing the detection speed by approximately 35%, shortening the single image processing time from 2.3 seconds to 1.5 seconds, improving the detection efficiency of the production line and helping to increase production capacity.

[0118] This system effectively enhances tolerance and adaptability to part offsets by introducing key area positioning based on a registration transformation matrix and an asymmetric tolerance band design. Combined with attention-guided image segmentation technology and an efficient convolutional neural network architecture, it improves overall system detection performance and robustness. In real-world production environments, the system maintains high accuracy and stability despite complex lighting changes and fine-tuning of part positions, significantly improving the automation level of battery box assembly quality inspection and meeting the needs of manufacturers' intelligent transformation.

[0119] The above content is merely an example and explanation of the structure of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should fall within the scope of protection of the present invention.

[0120] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0124] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0125] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for intelligently detecting the assembly status of product parts, characterized in that: include: S100: The servo module drives the industrial camera to move to the shooting point, captures images of a local area of ​​the battery box, and obtains a local image; S200: performing image registration on the local image and the corresponding template image to obtain a registration transformation matrix from the template image to the local image; S300: Calculate the offset of the parts assembly based on the registration transformation matrix and compare it with a preset offset threshold; If the offset is greater than the offset threshold, the part assembly state is judged to be unqualified; otherwise, the key detection area of ​​the template image is defined based on the offset, and reverse mapping is performed in combination with the inverse transformation of the registration transformation matrix to obtain the target detection area in the local image; S400: The target detection area is used as the attention-guiding area, and other areas are used as non-attention-guiding areas. An attention grayscale map is constructed, and the attention grayscale map is channel-joined with the local image. S500: performing image segmentation on the spliced ​​local image, dividing it into image segments and inputting them into the trained convolutional neural network model to perform classification detection of assembly anomalies; S600: Based on the output of the convolutional neural network, the detection results of each image segment are integrated to generate the assembly detection result of the local area and store it to the disk; S700: Repeat steps S100 to S600 until all areas of the battery box to be inspected are traversed, and generate a battery box assembly result through Boolean operations.

2. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The steps of obtaining the registration transformation matrix from the template image to the local image include: Extract salient feature points from the template image and the local image; Calculate matching pairs based on the feature points and use the RANSAC algorithm to eliminate abnormal matching pairs; The registration transformation matrix from the template image to the local image is obtained by fitting the affine transformation through the matching point set.

3. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The steps to calculate offsets for a part assembly include: Extract the centroid coordinates of the preset part area in the template image, and extract the actual centroid coordinates of the corresponding part in the local image; The centroid coordinates of the template are mapped to the local image coordinate system through the registration transformation matrix to obtain the theoretical centroid coordinates; Calculate the Euclidean distance and direction angle deviation between the theoretical centroid coordinates and the actual centroid coordinates.

4. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The steps of obtaining the target detection area in the local image include: Extract the part outline in the template image based on the edge detection algorithm; The calculated offset is asymmetrically expanded to generate a tolerance zone, and the key inspection area is generated in combination with the part contour; The key detection area is mapped to the local image coordinate system through the inverse transformation of the registration transformation matrix to obtain the corresponding target detection area.

5. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The operation of constructing the attention grayscale map is: The pixel values ​​in the target detection area are set to a preset high value, and the pixel values ​​in the remaining non-detection areas are set to a preset low value. The grayscale image is smoothed by Gaussian filtering to generate an attention grayscale image.

6. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The steps of performing image segmentation on the spliced ​​local image include: Binarize the attention grayscale image based on the preset grayscale threshold and extract the connected domain contour of the key detection area; Calculate the minimum bounding box for each connected area as the candidate area of ​​the image segment; The stitched local image is cropped using the candidate regions to obtain image segments.

7. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The convolutional neural network model is specifically ResNet, and the attention module used is the CBAM module.

8. The method for intelligently detecting the assembly status of product parts according to claim 1, characterized in that: The steps to generate the battery box assembly result through Boolean operations include: Obtain the assembly test results of each local area. Each test result is a binary state, indicating whether the assembly is qualified or unqualified; Perform a logical "OR" operation on the inspection results of all local areas. If any local area is judged to be unqualified, the overall assembly status is judged to be unqualified.

9. A system for intelligently detecting the assembly status of product parts, characterized in that: include: Image acquisition module, driven by the servo module to move the industrial camera to the shooting point, captures the image of the local area of ​​the battery box, and obtains a local image; The template matching module is used to perform image registration between the local image and the corresponding template image to obtain a registration transformation matrix from the template image to the local image; An offset determination module is used to calculate the offset of the part assembly based on the registration transformation matrix and compare it with a preset offset threshold; If the offset is greater than the offset threshold, the part assembly state is judged to be unqualified; otherwise, the key detection area of ​​the template image is defined based on the offset, and reverse mapping is performed in combination with the inverse transformation of the registration transformation matrix to obtain the target detection area in the local image; The channel augmentation module is used to construct an attention grayscale map with the target detection area as the attention-guiding area and other areas as the non-attention-guiding areas, and then perform channel splicing on the attention grayscale map and the local image; The anomaly detection module performs image segmentation on the spliced ​​local image, divides it into image segments, and inputs them into the trained convolutional neural network model to perform classification detection of assembly anomalies; The result analysis module is used to fuse the detection results of each image segment according to the output of the convolutional neural network, generate the assembly detection results of the local area and store them on disk; Repeat the above modules until all the areas to be inspected of the battery box are traversed, and the battery box assembly results are generated through Boolean operations.

Images