Stud welding detection method and device, electronic equipment and storage medium

Abstract

The invention provides a stud welding detection method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the tracking recognition processing of a stud welding region in a to-be-detected image based on a template frame, and obtaining a target welding region in the to-be-detected image; calculating a mapping relation matrix between the to-be-detected image and the template image, and mapping and aligning the template frame to a corresponding position of the to-be-detected image based on the mapping relation matrix to obtain a to-be-classified image area after mapping and alignment processing; and performing matching identification processing on the target welding area and the to-be-classified image area to obtain an identification classification result of the to-be-detected image. According to the embodiment of the invention, tracking identification is carried out by using the template frame, so that the target welding area in the to-be-detected image can be accurately positioned, and the accuracy of stud welding area detection is improved; the template frame can be accurately mapped and aligned to the corresponding position of the to-be-detected image; the alignment mode not only improves the processing speed, but also can improve the detection speed of the stud welding area.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a method and device for detecting stud welding, an electronic device, and a storage medium. Background Art

[0002] Stud welding, as an important connection technology in vehicle manufacturing, is widely used in the connection process of key components such as the frame and floor. As a commonly used connecting piece, studs are firmly connected to other vehicle components through welding to ensure the structural stability and safety of the entire vehicle. Studs may fall off during the stud welding process. This is usually caused by inaccurate stud positioning before welding, vibration during welding, or cooling shrinkage after welding. If a stud falls off, it will not only affect the structural stability of the entire vehicle, but may also cause problems in subsequent processes, such as the inability to tighten or assemble. Therefore, it is necessary to detect the presence of floor welding studs at the end of the vehicle welding production line.

[0003] While deep learning models can currently be used for identification, classification, and image alignment, this image processing method can detect the presence of studs in floor welds, it has significant drawbacks. First, its detection speed is relatively slow, making it difficult to meet the efficient production requirements of vehicle welding production lines. Second, this method requires extensive pre-training of the deep learning model, which increases the workload and time cost and can lead to unstable model performance due to limited training data. Therefore, optimizing stud weld detection methods by identifying weld area images has become a pressing issue. Summary of the Invention

[0004] The present disclosure provides a method and apparatus, electronic device, and storage medium for detecting stud welding. Its primary purpose is to improve the image recognition rate of the weld area when inspecting floor plate weld studs at the end of a vehicle assembly line, thereby optimizing the stud welding detection method.

[0005] According to a first aspect of the present disclosure, a method for detecting stud welding is provided, comprising:

[0006] Tracking and identifying the stud welding area in the image to be detected based on the template frame to obtain the target welding area in the image to be detected;

[0007] Calculating a mapping relationship matrix between the image to be detected and the template image, and mapping and aligning the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix to obtain an image region to be classified after mapping and alignment processing;

[0008] The target welding area is matched and identified with the area of ​​the image to be classified to obtain an identification and classification result of the image to be detected, and the stud welding missing area in the image to be detected is marked according to the identification and classification result.

[0009] In some embodiments, the tracking and identifying the stud welding area in the image to be detected based on the template frame to obtain the target welding area in the image to be detected includes:

[0010] Performing feature analysis on the image to be detected, and searching for a stud welding area that matches the template frame according to the analysis result;

[0011] The category information corresponding to the template frame is matched with the stud welding area to generate the target welding area.

[0012] In some embodiments, calculating a mapping relationship matrix between the image to be detected and the template image, and mapping and aligning the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix to obtain an image region to be classified after mapping and alignment processing, includes:

[0013] Performing feature matching processing on the image to be detected and the template image, and calculating a mapping relationship matrix between the image to be detected and the template image according to the matching processing result;

[0014] Based on the mapping relationship matrix, performing mapping transformation processing on the template frame;

[0015] The template frame after the mapping transformation is positioned in the image to be detected, and the image area to be classified corresponding to the template frame after the mapping transformation is determined.

[0016] In some embodiments, performing feature matching processing on the image to be detected and the template image, and calculating a mapping relationship matrix between the image to be detected and the template image according to the matching processing result, includes:

[0017] Scaling the image to be detected and the template image to a preset resolution size respectively;

[0018] Extracting feature points from the image to be detected and the template image after scaling processing, and performing matching processing based on the feature points to obtain matching point pairs;

[0019] Based on the matching point pairs, a mapping relationship matrix between the image to be detected and the template image is calculated.

[0020] In some embodiments, matching and identifying the target welding area with the image area to be classified to obtain the identification and classification result of the image to be detected includes:

[0021] Calculating a characteristic distance between the target welding area and the image area to be classified;

[0022] Based on the characteristic distance, matching processing is performed on the target welding area and the image area to be classified;

[0023] A target welding area in the image to be detected that does not match the area of ​​the image to be classified is identified, and an identification and classification result of the image to be detected is generated.

[0024] In some embodiments, the method further comprises:

[0025] Based on the recognition and classification results, determining whether the target vehicle body corresponding to the image to be detected passes the detection;

[0026] If the target vehicle body passes the inspection, the image to be inspected is updated to a template image and the image area to be classified is updated to a template frame.

[0027] According to a second aspect of the present disclosure, there is provided a device for detecting stud welding, comprising:

[0028] A recognition unit, configured to track and recognize the stud welding area in the image to be detected based on the template frame, so as to obtain a target welding area in the image to be detected;

[0029] an alignment unit, configured to calculate a mapping relationship matrix between the image to be detected and the template image, and align the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix, to obtain an image region to be classified after the mapping alignment processing;

[0030] The matching unit is used to match and identify the target welding area with the area of ​​the image to be classified to obtain an identification and classification result of the image to be detected, and mark the stud welding missing area in the image to be detected according to the identification and classification result.

[0031] In some embodiments, the identification unit includes:

[0032] An analysis module, configured to perform feature analysis on the image to be detected and search for a stud welding area that matches the template frame based on the analysis result;

[0033] A generating module is used to match the category information corresponding to the template frame with the stud welding area to generate the target welding area.

[0034] In some embodiments, the alignment unit includes:

[0035] a first calculation module, configured to perform feature matching processing on the image to be detected and the template image, and calculate a mapping relationship matrix between the image to be detected and the template image according to the matching processing result;

[0036] A transformation module, configured to perform mapping transformation processing on the template frame based on the mapping relationship matrix;

[0037] The determination module is used to locate the template frame after the mapping transformation in the image to be detected, and determine the image area to be classified corresponding to the template frame after the mapping transformation.

[0038] In some embodiments, the first computing module is further configured to:

[0039] Scaling the image to be detected and the template image to a preset resolution size respectively;

[0040] Extracting feature points from the image to be detected and the template image after scaling processing, and performing matching processing based on the feature points to obtain matching point pairs;

[0041] Based on the matching point pairs, a mapping relationship matrix between the image to be detected and the template image is calculated.

[0042] In some embodiments, the matching unit includes:

[0043] A second calculation module is used to calculate the characteristic distance between the target welding area and the image area to be classified;

[0044] A matching module, configured to perform matching processing on the target welding area and the image area to be classified based on the feature distance;

[0045] The generating module is used to identify the target welding area in the image to be detected that does not match the area of ​​the image to be classified, and generate the recognition and classification result of the image to be detected.

[0046] In some embodiments, the apparatus further comprises:

[0047] a judgment unit, configured to judge whether the target vehicle body corresponding to the image to be detected has passed the detection based on the recognition and classification result;

[0048] An updating unit is configured to update the image to be detected into a template image and update the image area to be classified into a template frame when the target vehicle body passes the detection.

[0049] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0050] at least one processor; and

[0051] a memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0053] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0054] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0055] The present disclosure provides a method and apparatus, electronic device and storage medium for detecting stud welding, which tracks and identifies the stud welding area in the image to be detected based on a template frame to obtain the target welding area in the image to be detected; calculates a mapping relationship matrix between the image to be detected and the template image, and maps and aligns the template frame to the corresponding position of the image to be detected based on the mapping relationship matrix to obtain the image area to be classified after the mapping and alignment processing; matches and identifies the target welding area with the image area to be classified to obtain the recognition and classification result of the image to be detected, and marks the stud welding missing area in the image to be detected based on the recognition and classification result. Compared with the related art, the embodiment of the present disclosure can accurately locate the target welding area in the image to be detected by using the template frame for tracking and identification, reduce the possibility of false detection and missed detection, and improve the accuracy of stud welding area detection; based on the mapping relationship matrix, the template frame can be accurately mapped and aligned to the corresponding position of the image to be detected, thereby quickly obtaining the image area to be classified; this alignment method not only improves the processing speed, but also realizes efficient alignment of the image to be detected and the template image, which can improve the detection speed of the stud welding area and optimize the detection method of the stud welding area.

[0056] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0058] Figure 1 A schematic flow chart of a method for detecting stud welding provided in an embodiment of the present disclosure;

[0059] Figure 2 A schematic flow chart of another stud welding detection method provided in an embodiment of the present disclosure;

[0060] Figure 3 A schematic diagram of a matching process;

[0061] Figure 4 A schematic structural diagram of a stud welding detection device provided in an embodiment of the present disclosure;

[0062] Figure 5 A schematic structural diagram of another stud welding detection device provided in an embodiment of the present disclosure;

[0063] Figure 6 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0064] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0065] The following describes a method and apparatus for stud welding detection, an electronic device, and a storage medium according to embodiments of the present disclosure with reference to the accompanying drawings.

[0066] Figure 1 A schematic flow chart of a method for detecting stud welding provided in an embodiment of the present disclosure.

[0067] like Figure 1 As shown, the method comprises the following steps:

[0068] Step 101 : Tracking and identifying the stud welding area in the image to be detected based on a template frame to obtain a target welding area in the image to be detected.

[0069] In an embodiment of the present disclosure, during the vehicle manufacturing process, in order to accurately detect whether stud welding has fallen off or missed, the presence of welding studs is detected at the end of the vehicle welding production line. First, it is necessary to capture images of the welding area of ​​the target vehicle body. This is usually done using a high-definition camera or professional image acquisition equipment. The captured image to be detected should contain sufficient details for subsequent processing and analysis. After the image of the welding area is captured, image enhancement processing is required. The purpose of image enhancement processing is to improve image quality and provide a good foundation for subsequent recognition.

[0070] Tracking and identifying the stud weld area in the image to be inspected is performed based on a template frame. The purpose is to determine the specific location of the stud weld in the image to be inspected. The template frame is a pre-sized frame created based on the stud weld area in a known standard weld image or an actual weld sample. During tracking and identification, a template frame encompassing the entire stud weld area is first selected. This template frame is used as a reference for searching and matching within the image to be inspected, either frame by frame or region by region. Tracking and identification can utilize, but is not limited to, image processing techniques and algorithms. Image algorithms analyze features in the image to be inspected, such as color, texture, and shape, and compare them with features in the template frame. This algorithm accurately locates the area matching the template frame, i.e., the target weld area. This process not only considers the overall image features but also focuses on details, ensuring accurate recognition results.

[0071] Step 102 : Calculate a mapping relationship matrix between the image to be detected and the template image, and align the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix to obtain an image region to be classified after mapping alignment processing.

[0072] In the disclosed embodiments, mapping alignment can eliminate any angular deviations, scale differences, or positional offsets between the image to be inspected, captured by a camera at the end of a vehicle welding production line, and the template image, thereby aligning the two in feature space. This significantly improves recognition accuracy and reliability during subsequent image processing and analysis. Through mapping alignment, the information in the template frame can be accurately mapped to the image to be inspected, enabling precise positioning and classification of the target area.

[0073] Recognition algorithms are used to extract key feature points from the image to be inspected and the template image. These feature points are typically prominent and stable areas in the image, such as corners and edges, and they remain consistent across different images. By comparing these feature points, the correspondence between the two images can be found, that is, which feature points match in the two images. Based on these matched feature point pairs, mathematical algorithms can be used to estimate the mapping relationship matrix between the two images. The mapping relationship matrix describes the transformation process from the template image to the image to be inspected, including possible transformations such as translation, rotation, and scaling. After obtaining the mapping relationship matrix, the template frame can be mapped and aligned to the corresponding position in the image to be inspected. This process effectively transforms each point in the template frame using the mapping relationship matrix so that it finds the correct corresponding position in the image to be inspected. In this way, the template frame is accurately placed on the target weld area in the image to be inspected.

[0074] Step 103 : matching and identifying the target welding area with the area of ​​the image to be classified to obtain an identification and classification result of the image to be detected, and marking the stud welding missing area in the image to be detected according to the identification and classification result.

[0075] In an embodiment of the present disclosure, during the matching and recognition process, feature information of the target welding area and the image area to be classified is extracted. These features may include various attributes such as color, texture, and shape. Using the image matching algorithm, these feature information are compared and analyzed to determine the similarities and differences between them. Through the matching and recognition process, the recognition and classification results of the image to be detected can be obtained. These results usually include multiple categories such as normal welding, missing welding, and poor welding. For each category, we will set corresponding judgment criteria and thresholds to ensure the accuracy and reliability of the classification. The missing stud welding area in the image to be detected can be marked according to the recognition and classification results. The marking process usually uses eye-catching colors or symbols to clearly identify the missing area in the image for subsequent analysis and processing. In this way, the operator can intuitively understand the welding quality, discover and repair problems in a timely manner, thereby improving production efficiency and product quality.

[0076] The present disclosure provides a method for detecting stud welding, which tracks and identifies the stud welding area in the image to be detected based on a template frame to obtain the target welding area in the image to be detected; calculates a mapping relationship matrix between the image to be detected and the template image, and maps and aligns the template frame to the corresponding position of the image to be detected based on the mapping relationship matrix to obtain the image area to be classified after the mapping alignment processing; matches and identifies the target welding area with the image area to be classified to obtain the recognition and classification result of the image to be detected, and marks the stud welding missing area in the image to be detected based on the recognition and classification result. Compared with the related art, the embodiment of the present disclosure can accurately locate the target welding area in the image to be detected by using the template frame for tracking and identification, reduce the possibility of false detection and missed detection, and improve the accuracy of stud welding area detection; based on the mapping relationship matrix, the template frame can be accurately mapped and aligned to the corresponding position of the image to be detected, thereby quickly obtaining the image area to be classified; this alignment method not only improves the processing speed, but also realizes efficient alignment of the image to be detected and the template image, thereby improving the detection speed of the stud welding area.

[0077] In order to clearly illustrate the embodiments of the present disclosure, this embodiment provides a flow chart of another method for detecting stud welding.

[0078] like Figure 2 As shown, the method comprises the following steps:

[0079] Step 201 : performing feature analysis on the image to be detected, and searching for a stud welding area matching the template frame according to the analysis result.

[0080] Step 202 : Match the category information corresponding to the template frame with the stud welding area to generate the target welding area.

[0081] Specifically, in steps 201 to 202, the image to be detected and the template frame are input into the CSR (Channel Spatial Reliability) tracker to track and identify the stud welding area in the image to be detected. The tracker will find the target object (target welding area) that matches the template frame in the subsequent image to be detected. It predicts the position of the target object in the image to be detected by analyzing the color, texture and other features in the image and combining the spatial position information. By analyzing the features in the image to be detected, the area that matches the template frame, that is, the target welding area, can be accurately found. This process not only takes into account the overall characteristics of the image, but also pays attention to the details, which can ensure the accuracy of the recognition results.

[0082] Step 203 : performing feature matching processing on the image to be detected and the template image, and calculating a mapping relationship matrix between the image to be detected and the template image according to the matching processing result.

[0083] As an implementable method of the embodiment of the present disclosure, when performing feature matching processing on the image to be detected and the template image, the following steps may be adopted but are not limited to:

[0084] Step 2031: scaling the image to be detected and the template image to a preset resolution size respectively.

[0085] Step 2032 : extracting feature points from the image to be detected and the template image after the scaling process, and performing matching processing based on the feature points to obtain matching point pairs.

[0086] Step 2033: Calculate a mapping relationship matrix between the image to be detected and the template image based on the matching point pairs.

[0087] Specifically, in step 203, the image to be detected and the template image are adjusted to a uniform and appropriate resolution. This helps reduce the computational effort during subsequent image processing while ensuring the clarity and extractability of image features. Typically, a preset resolution is set based on the actual application scenario and hardware performance to optimize processing speed while preserving image detail.

[0088] After scaling the image to be detected and the template image to the preset resolution, feature points need to be extracted. Feature points are unique and significant local areas in an image, such as corners, edges, or areas with noticeable texture changes. These feature points are stable during image transformation and matching, and can establish correspondence between images.

[0089] In the feature point extraction stage, algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), or ORB (Oriented BRIEF and Rotation Invariant) can be used, but are not limited to these algorithms. These algorithms can automatically detect key points in the image and generate corresponding feature descriptors for subsequent matching.

[0090] After obtaining the feature points, matching is performed based on them. The goal of matching is to find corresponding pairs of matching points in the image to be detected and the template image. This is typically achieved by calculating the similarity or distance between the matching point pairs, such as the Euclidean distance or Hamming distance. By setting an appropriate matching threshold, we can filter out matching point pairs with a high degree of matching, which serves as the basis for subsequent calculation of the mapping relationship matrix.

[0091] Based on the matching point pairs, mathematical methods and optimization algorithms are used to calculate the homography change matrix (mapping relationship matrix) between the image to be inspected and the template image. This matrix describes the mapping relationship between the template image and the image to be inspected. By calculating the mapping relationship matrix, the correspondence between the image to be inspected and the template image can be accurately established, providing strong support for subsequent identification and classification of the target weld area.

[0092] Step 204: Perform mapping transformation processing on the template frame based on the mapping relationship matrix.

[0093] Step 205 : Positioning the template frame after the mapping transformation in the image to be detected, and determining the image region to be classified corresponding to the template frame after the mapping transformation.

[0094] Specifically, in steps 204 and 205, after calculating the homography transformation matrix H, the template frame can be mapped from the template image space to the image space to be detected. The mapped frame corresponds to the template frame's position in the image to be detected. Using this mapped frame, we can locate the region in the image to be detected that corresponds to the template image, thereby obtaining the image region to be classified.

[0095] For example, define the coordinates of a template frame as (x1, y1, x2, y2), and the homography matrix is

[0096] The transformed template frame coordinates are (x1 ′,y1 ′ ,x2 ′ ,y2 ′ ),but

[0097] Step 206: Calculate the characteristic distance between the target welding area and the image area to be classified.

[0098] Step 207 : performing matching processing on the target welding area and the image area to be classified based on the feature distance.

[0099] Step 208 : Identify the target welding area in the image to be detected that does not match the area of ​​the image to be classified, and generate a recognition and classification result of the image to be detected.

[0100] Specifically, in steps 206 to 208, the aligned template frame is matched with the tracked target welding area based on distance, as shown in formula (1), where (xy) 2 represents the square of the distance between a certain aligned template frame and the tracked target welding area. Therefore, the goal of formula (1) is to find the best matching pair by minimizing the distance between all frames. This module can filter out incorrect targets, reduce false positives, and obtain the final positioning and recognition results in the image to be detected. Figure 3 This is a schematic diagram of the matching process. The tracked box in the figure is the target welding area, and the aligned template box is the image area to be classified. The positioning and recognition results are obtained by matching the tracked box with the aligned template box.

[0101]

[0102] Step 209 : marking the stud welding missing area in the image to be detected according to the recognition and classification result.

[0103] Specifically, in step 209, the original image to be detected is marked according to the recognition and classification results. For each recognition result, it is matched with the original image to be detected. If a certain recognition result indicates that a stud is detected in the image, a mark is made at the corresponding position. This mark can be a color mark, a shape mark, or a text annotation, etc., depending on the needs of the actual application. By marking directly in the image to be detected, it is convenient for the staff to have a more comprehensive understanding of the situation of the stud and provide a reference for subsequent operations. It should be noted that the above-mentioned marking method is only exemplary and does not limit how to mark the welding area image in this disclosure.

[0104] Step 210 : Based on the recognition and classification result, it is determined whether the target vehicle body corresponding to the image to be detected has passed the detection.

[0105] Step 211 : When the target vehicle body passes the inspection, the image to be inspected is updated to a template image and the image area to be classified is updated to a template frame.

[0106] Specifically, in steps 210 to 211, based on the recognition and classification results obtained in the aforementioned steps, it can be further determined whether the target vehicle body corresponding to the image to be inspected has successfully passed the quality inspection. If the target vehicle body has passed the inspection, that is, there are no obvious defects or missing parts in its stud welding area, then it can be considered that the welding quality of this vehicle is qualified. At this time, in order to further improve the accuracy and adaptability of the detection system, the image to be inspected that has currently passed the inspection can be updated to a new template image. The advantage of doing so is that as time goes by and the production environment changes, the new template image can better reflect the actual situation of stud welding under current production conditions, thereby improving the accuracy of subsequent inspections. At the same time, the image area to be classified is also updated to a new template frame. The template frame is a key tool for feature matching and mapping alignment. By continuously updating the template frame, it can be ensured that the subsequent inspection process can more accurately locate the stud welding area, further improving the efficiency and accuracy of the inspection.

[0107] In summary, the embodiments of the present disclosure have the following beneficial effects:

[0108] 1. There is no need to conduct extensive training of deep learning models in advance, which reduces workload and time cost, and avoids the instability of model performance due to the limitations of training data.

[0109] 2. By mapping and aligning the template frame with the image to be inspected and the template image, rather than aligning the image to be inspected with the template frame, this alignment method can accurately map the template frame to the corresponding position of the image to be inspected, thereby quickly obtaining the image area to be classified. This not only improves processing speed and achieves efficient alignment of the image to be inspected and the template image, but also can improve the inspection speed of the stud welding area.

[0110] It should be noted that the embodiments of the present disclosure may include multiple steps. For the convenience of description, these steps are numbered, but these numbers do not limit the execution time slots or execution order between the steps; these steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0111] Corresponding to the above-mentioned stud welding detection method, the present invention also provides a stud welding detection device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, any details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment and will not be repeated in this invention.

[0112] Figure 4 A schematic diagram of the structure of a stud welding detection device provided in an embodiment of the present disclosure is shown as follows: Figure 4 Shown, including:

[0113] The recognition unit 31 is configured to track and recognize the stud welding area in the image to be detected based on the template frame to obtain the target welding area in the image to be detected;

[0114] an alignment unit 32 for calculating a mapping relationship matrix between the image to be detected and the template image, and mapping and aligning the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix, to obtain an image region to be classified after mapping and alignment processing;

[0115] The matching unit 33 is configured to perform matching and recognition processing on the target welding area and the area of ​​the image to be classified to obtain a recognition and classification result of the image to be detected, and mark the stud welding missing area in the image to be detected according to the recognition and classification result.

[0116] The present disclosure provides a device for stud welding detection, which tracks and identifies the stud welding area in the image to be detected based on a template frame to obtain a target welding area in the image to be detected; calculates a mapping relationship matrix between the image to be detected and the template image, and maps and aligns the template frame to the corresponding position of the image to be detected based on the mapping relationship matrix to obtain the image area to be classified after the mapping alignment processing; matches and identifies the target welding area with the image area to be classified to obtain a recognition and classification result of the image to be detected, and marks the stud welding missing area in the image to be detected based on the recognition and classification result. Compared with the related art, the embodiment of the present disclosure can accurately locate the target welding area in the image to be detected by using the template frame for tracking and identification, reducing the possibility of false detection and missed detection, and improving the accuracy of stud welding area detection; based on the mapping relationship matrix, the template frame can be accurately mapped and aligned to the corresponding position of the image to be detected, thereby quickly obtaining the image area to be classified; this alignment method not only improves the processing speed, but also realizes efficient alignment of the image to be detected and the template image, thereby improving the detection speed of the stud welding area.

[0117] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the identification unit 31 includes:

[0118] An analysis module 311 is configured to perform feature analysis on the image to be detected and search for a stud welding area that matches the template frame based on the analysis result;

[0119] The generating module 312 is configured to match the category information corresponding to the template frame with the stud welding area to generate the target welding area.

[0120] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the alignment unit 32 includes:

[0121] A first calculation module 321 is configured to perform feature matching processing on the image to be detected and the template image, and calculate a mapping relationship matrix between the image to be detected and the template image based on the matching processing result;

[0122] A transformation module 322 is configured to perform mapping transformation processing on the template frame based on the mapping relationship matrix;

[0123] The determination module 323 is configured to locate the template frame after the mapping transformation in the image to be detected, and determine the image region to be classified corresponding to the template frame after the mapping transformation.

[0124] Furthermore, in a possible implementation of this embodiment, the first calculation module 321 is further configured to:

[0125] Scaling the image to be detected and the template image to a preset resolution size respectively;

[0126] Extracting feature points from the image to be detected and the template image after scaling processing, and performing matching processing based on the feature points to obtain matching point pairs;

[0127] Based on the matching point pairs, a mapping relationship matrix between the image to be detected and the template image is calculated.

[0128] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the matching unit 33 includes:

[0129] A second calculation module 331 is used to calculate the characteristic distance between the target welding area and the image area to be classified;

[0130] A matching module 332 is configured to perform matching processing on the target welding area and the image area to be classified based on the feature distance;

[0131] The generating module 333 is configured to identify a target welding area in the image to be detected that does not match the area of ​​the image to be classified, and generate a recognition and classification result of the image to be detected.

[0132] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the device also includes:

[0133] A judgment unit 34 is configured to judge whether the target vehicle body corresponding to the image to be detected has passed the detection based on the recognition and classification result;

[0134] The updating unit 35 is configured to update the image to be detected into a template image and update the image region to be classified into a template frame when the target vehicle body passes the detection.

[0135] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principles are the same, which is not limited in this embodiment.

[0136] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0137] Figure 6 A schematic block diagram of an example electronic device 400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0138] like Figure 6 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 into a RAM (Random Access Memory) 403. Various programs and data required for the operation of the device 400 can also be stored in the RAM 403. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0139] Various components in device 400 are connected to I / O interface 405, including an input unit 406, such as a keyboard, mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless communication transceiver, etc. Communication unit 409 allows device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0140] The computing unit 401 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), various specialized AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the stud weld inspection method. For example, in some embodiments, the stud weld inspection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned stud welding detection method in any other appropriate manner (for example, by means of firmware).

[0141] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0146] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0147] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0148] The various numerical numbers such as first and second involved in the present disclosure are only for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, and also indicate the order of precedence.

[0149] The at least one in the present disclosure can also be described as one or more, and the multiple can be two, three, four or more, which is not limited in the present disclosure. In the embodiments of the present disclosure, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", and there is no order of precedence or size between the technical features described by "first", "second", "third", "A", "B", "C" and "D".

[0150] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0151] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for detecting stud welding, characterized in that: include: Tracking and identifying the stud welding area in the image to be detected based on the template frame to obtain the target welding area in the image to be detected; Calculating a mapping relationship matrix between the image to be detected and the template image, and mapping and aligning the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix to obtain an image region to be classified after mapping and alignment processing; The target welding area is matched and identified with the area of ​​the image to be classified to obtain an identification and classification result of the image to be detected, and the stud welding missing area in the image to be detected is marked according to the identification and classification result.

2. The method according to claim 1, characterized in that The tracking and identifying process of the stud welding area in the image to be detected based on the template frame to obtain the target welding area in the image to be detected includes: Performing feature analysis on the image to be detected, and searching for a stud welding area that matches the template frame according to the analysis result; The category information corresponding to the template frame is matched with the stud welding area to generate the target welding area.

3. The method according to claim 1, characterized in that The calculating of the mapping relationship matrix between the image to be detected and the template image, and mapping and aligning the template frame to the corresponding position of the image to be detected based on the mapping relationship matrix to obtain the image area to be classified after the mapping and alignment processing, includes: Performing feature matching processing on the image to be detected and the template image, and calculating a mapping relationship matrix between the image to be detected and the template image according to the matching processing result; Based on the mapping relationship matrix, performing mapping transformation processing on the template frame; The template frame after the mapping transformation is positioned in the image to be detected, and the image area to be classified corresponding to the template frame after the mapping transformation is determined.

4. The method according to claim 3, characterized in that The step of performing feature matching processing on the image to be detected and the template image, and calculating a mapping relationship matrix between the image to be detected and the template image according to the result of the matching processing, includes: Scaling the image to be detected and the template image to a preset resolution size respectively; Extracting feature points from the image to be detected and the template image after scaling processing, and performing matching processing based on the feature points to obtain matching point pairs; Based on the matching point pairs, a mapping relationship matrix between the image to be detected and the template image is calculated.

5. The method according to claim 1, wherein The matching and identification processing of the target welding area and the image area to be classified to obtain the identification and classification result of the image to be detected includes: Calculating a characteristic distance between the target welding area and the image area to be classified; Based on the characteristic distance, matching processing is performed on the target welding area and the image area to be classified; A target welding area in the image to be detected that does not match the area of ​​the image to be classified is identified, and an identification and classification result of the image to be detected is generated.

6. The method according to claim 1, characterized in that The method further comprises: Based on the recognition and classification results, determining whether the target vehicle body corresponding to the image to be detected passes the detection; If the target vehicle body passes the inspection, the image to be inspected is updated to a template image and the image area to be classified is updated to a template frame.

7. A stud welding detection device, characterized in that: include: A recognition unit, configured to track and recognize the stud welding area in the image to be detected based on the template frame, so as to obtain a target welding area in the image to be detected; an alignment unit, configured to calculate a mapping relationship matrix between the image to be detected and the template image, and align the template frame to a corresponding position of the image to be detected based on the mapping relationship matrix, to obtain an image region to be classified after the mapping alignment processing; The matching unit is used to match and identify the target welding area with the area of ​​the image to be classified to obtain an identification and classification result of the image to be detected, and mark the stud welding missing area in the image to be detected according to the identification and classification result.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images