Body-in-white spot welding quality detection method and device, terminal and medium

Through the improved YOLOv8s algorithm and SIFT matching algorithm, the detection problem of similar shapes of the body-white welding joints is solved, and the rapid and accurate detection of the quality of the welding joints is achieved, the detection accuracy and efficiency are improved, and human error is avoided.

CN119048479APending Publication Date: 2024-11-29CHINA FAW CO LTD

Patent Information

Application Number
CN202411190814.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In the production of body white, the shape of the solder joint is similar to the shape of the hole position, which makes it difficult to distinguish manual inspection, which may lead to missed or missed inspection. The existing detection methods are inefficient and have artificial subjective errors.

Method used

The improved YOLOv8s algorithm is used to acquire and preprocess the solder joint images, and the target detection is performed using the YOLOv8s-FAC network model, and the solder joint quality is judged in combination with the SIFT matching algorithm to achieve fast and accurate detection.

Benefits of technology

Pixel-level solder joint detection is realized, avoiding human subjective errors, improving detection accuracy and efficiency, and reducing missed and missed detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048479A_ABST
    Figure CN119048479A_ABST
Patent Text Reader

Abstract

The invention discloses a body-in-white spot welding quality detection method and device, a terminal and a medium, and belongs to the technical field of body-in-white production, and the body-in-white spot welding quality detection method comprises the following steps: obtaining a welding spot image collected by a camera, and preprocessing the welding spot image to obtain a plurality of processed welding spot images; the processed welding spot image is imported into an optimal YOLOv8s-FAC network model, and a marked welding spot image is obtained; carrying out interception processing on the marked welding spot images to obtain a plurality of intercepted welding spot images, and storing the intercepted welding spot images in a specified folder; traversing a plurality of intercepted welding spot images in the specified folder and carrying out template matching on the intercepted welding spot images and high-quality welding spot images in a database to obtain scale factors corresponding to a plurality of feature points of the welding spots of the body in white in the images; and judging the quality of the body-in-white welding spots by adopting an SIFT matching algorithm for the scale factors corresponding to the plurality of feature points of the body-in-white welding spots in the plurality of images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention discloses a body-in-white spot welding quality detection method, device, terminal and medium, and belongs to the technical field of body-in-white production. Background Art

[0002] In the automotive industry, welding quality directly affects the quality and safety of the entire vehicle. To ensure the structural integrity and functionality of the welds, rapid and accurate detection of weld quality is a key quality control step in the automotive welding process. In the body-in-white production process, defects such as overburning or cold welding will lead to unstable welds, thereby reducing the mechanical properties of the body structure, reducing the service life of the vehicle, and posing certain safety hazards. Therefore, real-time and accurate detection of weld quality can effectively troubleshoot possible welding defects in the production process, and is an important measure to ensure the safety of the body structure, improve product quality, and reduce later costs.

[0003] At present, the methods for welding spot quality inspection are mainly divided into contact and non-contact methods. Many companies mainly use manual sampling to judge the quality of the welding spots of the car body by visual observation and touch. This inspection method is not only complicated in operation process, but also easily interfered by ambient light, rust and stains, resulting in subjective errors in welding quality inspection results, low inspection efficiency, and difficulty in meeting the needs of batch inspection, with high labor and time costs. Summary of the invention

[0004] In view of the defects of the prior art, the present invention proposes a body-in-white spot welding quality detection method, device, terminal and medium to solve the problem that in the white body weld quality detection, the shape of the weld is similar to the shape of some holes, which is difficult to distinguish during manual detection, which may lead to missed detection and false detection. Therefore, by using the improved yolov8s algorithm, real-time weld quality detection can be performed to avoid misjudgment of welds, thereby achieving rapid and accurate detection of weld quality.

[0005] The technical solution of the present invention is as follows:

[0006] According to a first aspect of an embodiment of the present invention, a method for detecting the quality of spot welding of a body in white is provided, comprising:

[0007] Acquire the welding spot images captured by the camera, and pre-process the welding spot images to obtain a plurality of processed welding spot images;

[0008] Importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image;

[0009] The marked welding spot images are intercepted to obtain a plurality of intercepted welding spot images and saved in a designated folder;

[0010] The multiple intercepted welding point images in the designated folder are traversed and template matching is performed with the high-quality welding point images in the database to obtain the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images, and the SIFT matching algorithm is used to judge the quality of the body-in-white welding points for the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images.

[0011] Preferably, before obtaining the welding spot image captured by the camera, the method further includes: obtaining a hand-eye calibration matrix, and controlling the camera at the end of the manipulator until the welding spot is accurately aligned to the center of the camera's field of view.

[0012] Preferably, before obtaining the hand-eye calibration matrix, the method further includes:

[0013] Collecting a plurality of high-quality body-in-white weld spot image data, processing and annotating the plurality of high-quality body-in-white weld spot image data to obtain a high-quality body-in-white weld spot image data set;

[0014] A YOLOv8s-FAC network model for detecting the quality of body-in-white welds is constructed, and the YOLOv8s-FAC network model is trained using the high-quality body-in-white weld image dataset to obtain the optimal YOLOv8s-FAC network model.

[0015] Preferably, the preprocessing the weld spot image to obtain a processed weld spot image includes:

[0016] Bilateral filtering is used to reduce noise in the solder joint image to remove noise information from the image, and the image is converted into the HSV color space. Feature enhancement is performed through the histogram, and data enhancement is performed on the image after feature enhancement. The image size is adjusted to the expected input size of the optimal YOLOv8s-FAC network model to obtain the processed solder joint image.

[0017] Preferably, the step of importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image comprises:

[0018] The processed solder joint image is imported into the optimal YOLOv8s-FAC network model, the dilation rate is adjusted according to the image to capture the image details, the convolution kernel size is adaptively adjusted according to the image content, the feature information of different frequencies is balanced in the image, and the size of the feature map is gradually reduced and the number of channels is increased to obtain the high-level feature information of the image;

[0019] Inputting high-level feature information of the image into the head network of the YOLOv8s-FAC network, generating original predictions for target detection of the image through three detection heads, converting the original predictions of target detection of the image into coordinates and sizes of bounding boxes through feature maps output by the network, and converting probability density distribution into class probabilities to obtain solder joint images with multiple bounding box coordinates, class probabilities and confidence scores;

[0020] Traversing the solder joint images with multiple bounding box coordinates, category probabilities and confidence scores, filtering out bounding boxes with confidence scores lower than a confidence threshold, and obtaining solder joint images with multiple final bounding box coordinates, category labels and confidence scores, wherein the solder joint images with multiple final bounding box coordinates, category labels and confidence scores are multiple marked solder joint images.

[0021] Preferably, the intercepting the marked welding spot image to obtain a plurality of intercepted welding spot images and saving them in a designated folder comprises:

[0022] The marked multiple welding spot images are intercepted by image clipping technology according to the final bounding box coordinates of each bounding box to obtain the intercepted multiple welding spot images;

[0023] The captured multiple welding spot images are saved in a designated folder.

[0024] According to a second aspect of an embodiment of the present invention, there is provided a body-in-white spot welding quality inspection device, comprising:

[0025] An acquisition module is used to acquire the welding point images collected by the camera, and pre-process the welding point images to obtain a plurality of processed welding point images;

[0026] A marking module, used for importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image;

[0027] A capture module, used for performing capture processing on the marked welding spot image to obtain a plurality of captured welding spot images and save them into a designated folder;

[0028] The judging module is used to traverse the multiple intercepted welding point images in the specified folder and perform template matching with the high-quality welding point images in the database to obtain the corresponding scale factors of multiple feature points of the body-in-white welding points in the multiple images, and use the SIFT matching algorithm to judge the quality of the body-in-white welding points on the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images.

[0029] Preferably, the marking module is used to:

[0030] The processed solder joint image is imported into the optimal YOLOv8s-FAC network model, the dilation rate is adjusted according to the image to capture the image details, the convolution kernel size is adaptively adjusted according to the image content, the feature information of different frequencies is balanced in the image, and the size of the feature map is gradually reduced and the number of channels is increased to obtain the high-level feature information of the image;

[0031] Inputting high-level feature information of the image into the head network of the YOLOv8s-FAC network, generating original predictions for target detection of the image through three detection heads, converting the original predictions of target detection of the image into coordinates and sizes of bounding boxes through feature maps output by the network, and converting probability density distribution into class probabilities to obtain solder joint images with multiple bounding box coordinates, class probabilities and confidence scores;

[0032] Traversing the solder joint images with multiple bounding box coordinates, category probabilities and confidence scores, filtering out bounding boxes with confidence scores lower than a confidence threshold, and obtaining solder joint images with multiple final bounding box coordinates, category labels and confidence scores, wherein the solder joint images with multiple final bounding box coordinates, category labels and confidence scores are multiple marked solder joint images.

[0033] According to a third aspect of an embodiment of the present invention, a terminal is provided, including:

[0034] one or more processors;

[0035] a memory for storing the one or more processor-executable instructions;

[0036] Wherein, the one or more processors are configured to:

[0037] Execute the method described in the first aspect of the embodiment of the present invention.

[0038] According to a fourth aspect of an embodiment of the present invention, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the method described in the first aspect of the embodiment of the present invention.

[0039] According to a fifth aspect of an embodiment of the present invention, an application product is provided. When the application product is running on a terminal, the terminal executes the method described in the first aspect of the embodiment of the present invention.

[0040] The beneficial effects of the present invention are:

[0041] The present invention provides a body-in-white spot welding quality detection method, device, terminal and medium, which can quickly detect the quality of welding spots, can perform pixel-level detection and measurement, avoid human subjective errors, and have higher detection accuracy and higher detection efficiency.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a flow chart of a method for inspecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0044] Figure 2 The figure is a flow chart of a method for inspecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0045] Figure 3 The diagram is a schematic diagram of the layout of detection equipment in a method for detecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0046] Figure 4 The present invention is a schematic diagram showing a hand-eye calibration principle in a method for inspecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0047] Figure 5 It is a structure diagram of a YOLOv8s-FAC network model in a method for detecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0048] Figure 6 It is a FAC structure diagram in a method for detecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0049] Figure 7 1 is an improved Bottleneck diagram in a method for detecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0050] Figure 8 The diagram is a schematic diagram of a template matching principle in a method for detecting the quality of spot welding of a body in white according to an exemplary embodiment.

[0051] Fig. 9 A schematic block diagram of the structure of a body-in-white spot welding quality inspection device is shown according to an exemplary embodiment.

[0052] Fig.10 A schematic block diagram of a terminal structure is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0055] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] An embodiment of the present invention provides a method for detecting the quality of spot welding of a body in white. The method is implemented by a terminal, and the terminal at least includes a CPU and the like.

[0057] Embodiment 1

[0058] Figure 1 and 2 The flowchart of a method for detecting the quality of spot welding of a body in white according to an exemplary embodiment is shown. The method is used in a terminal and comprises the following steps:

[0059] Step 101, obtaining the welding point images captured by the camera, and preprocessing the welding point images to obtain a plurality of processed welding point images, the specific contents are as follows:

[0060] like Figure 3-7 As shown, before acquiring the welding spot image collected by the camera, it also includes:

[0061] Get the hand-eye calibration matrix and control the camera at the end of the robot until the weld point is accurately aligned to the center of the camera's field of view. The details are as follows:

[0062] Before obtaining the hand-eye calibration matrix, the method further includes: collecting high-quality solder joint images with smooth surfaces and no defects such as re-welding, burrs, and spatters and making them into a database to provide a judgment standard for subsequent solder joint quality assessment;

[0063] Before obtaining the hand-eye calibration matrix, it also includes: collecting multiple high-quality body-in-white weld spot image data, processing and annotating the multiple high-quality body-in-white weld spot image data to obtain a high-quality body-in-white weld spot image data set. Construct a YOLOv8s-FAC network model for detecting the quality of body-in-white weld spots, use a high-quality body-in-white weld spot image data set divided into a training set, a validation set, and a test set in a ratio of 7:2:1, and train the YOLOv8s-FAC network model to obtain the best YOLOv8s-FAC network model.

[0064] Then the body-in-white to be tested moves to the testing position, and according to the known camera external parameter conversion matrix T1, a certain point P0 in the body-in-white coordinate system is converted to point P1 in the camera coordinate system, and then according to the desired hand-eye calibration matrix X, it is converted to point P2 in the robot end coordinate system, and then combined with the known robot self parameter conversion matrix T3, it can be converted to point P3 in the robot base coordinate system. Therefore, the relationship can be obtained as follows:

[0065] T 3 XT 1 P 0 =P 3 (1)

[0066] For the same point in the body-in-white coordinate system, the values ​​of P0 and P3 remain unchanged, but the values ​​of P1 and P2 change. The above relationship becomes as follows:

[0067] T 3 'XT 1 'P 0 =P 3 (2)

[0068] Where T 3 ' and T 1 ' is also a known parameter of the second measurement. Combining (1) and (2), we can get the following relationship:

[0069] T 3 XT 1 =T 3 'XT 1 ' (3)

[0070] By transforming formula (3), we can get:

[0071] T 3 ' -1 T 3 X=XT 1 'T 1 -1 (4)

[0072] Formula (4) can be viewed as an equation of AX = XB, and the matrix A = T 3 ' -1 T 3 , B = T 1 'T 1 -1 By solving the calibration equation AX=XB, we can get the hand-eye calibration matrix X, and then complete the precise positioning of the solder joint.

[0073] Through the hand-eye calibration matrix X, the coordinates of the weld points to be tested in the body-in-white can be obtained. i0 (i=1,2,3,…,n-1,n), converted to the coordinate point P in the robot coordinate system i2 (i=1,2,3,…,n-1,n), move the end of the robot to P i2 point, thus moving the camera to the top of the body-in-white welding point, so that the camera continuously monitors the welding point position and provides real-time feedback on the adjusted result until the welding point is accurately aligned to the center of the camera's field of view and the welding point image can be clearly imaged on the image acquisition card;

[0074] If some welding spots are distributed on different planes, calibrate the different planes respectively and repeat the above steps to make the welding spots in the center of the image and have a clear image.

[0075] The solder joint image captured by the camera is obtained, and the noise information of the image is filtered out by bilateral filtering. The image is converted to HSV color space, and the feature is enhanced through the histogram. The image after feature enhancement is data enhanced, and the image size is adjusted to the input size expected by the optimal YOLOv8s-FAC network model to obtain the processed solder joint image. The image size is preferably 640*640.

[0076] Step 102, importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image, the specific steps are as follows:

[0077] Import the processed weld image into the optimal YOLOv8s-FAC network model, adjust the dilation rate according to the image; optimize the coverage of different frequency components to capture image details; adaptively adjust the convolution kernel size according to the image content, thereby emphasizing the feature information of different frequencies and improving the robustness and generalization ability of the model; balance the feature information of different frequencies in the image, and improve the overall model performance by focusing on related features. And gradually reduce the size of the feature map and increase the number of channels to obtain the high-level feature information of the image. The above frequency-adaptive dilated convolution improves the network's ability to extract features, improves the network detection performance, speeds up the network detection speed, and ensures the real-time and accuracy of the body-in-white weld detection.

[0078] The high-level feature information of the image is input into the head network of the YOLOv8s-FAC network, and the original predictions for target detection of the image are generated through the three detection heads. The original predictions of target detection of the image are converted into the coordinates and size of the bounding box through the feature map output by the network, and the probability density distribution is converted into the category probability to obtain solder joint images with multiple bounding box coordinates, category probabilities and confidence scores.

[0079] Traverse the solder joint images with multiple bounding box coordinates, category probabilities and confidence scores to filter out the bounding boxes with confidence scores below the confidence threshold to obtain solder joint images with multiple final bounding box coordinates, category labels and confidence scores, that is, for each category, apply non-maximum suppression to remove bounding boxes with high overlap, thereby retaining the bounding box with the highest confidence score and eliminating other bounding boxes with an intersection-over-union ratio higher than the threshold. The solder joint images with multiple final bounding box coordinates, category labels and confidence scores are multiple labeled solder joint images.

[0080] Step 103, intercepting the marked welding spot image to obtain a plurality of intercepted welding spot images and saving them to a designated folder, the specific contents are as follows:

[0081] The marked multiple welding spot images are intercepted and processed by image cropping technology according to the final bounding box coordinates of each bounding box to obtain the intercepted multiple welding spot images, and the intercepted multiple welding spot images are saved in a designated folder. The specific steps are as follows:

[0082] After the image is output by the network, each solder joint image with a confidence score exceeding the set threshold will be selected on the input image. Assuming that n targets are detected in an image, through image cropping technology, each pixel is traversed to find the upper left point (x 1i ,y 1i )(i∈(1,n)), and each bounding box corresponds to a set of (w i ,h i ), w i is the width of the boundingbox, h i is the height of the boundingbox, calculated according to the following formula:

[0083] x 2i =x 1i +w i (5)

[0084] y 2i =y 1i +h i (6)

[0085] Get the lower right point (x) of each bounding box 2i ,y2i ), according to the upper left point (x 1i ,y 1i ) and the lower right point (x 2i ,y 2i ) is the area occupied by each bounding box, so that each target is completely cut out from the image. Save it to the specified folder. The number of images in the folder is the number of solder joints detected. Compare the number of images in the folder with the number of solder joints required by the solder joint process to determine whether there is any leakage.

[0086] Step 104, traverse the multiple intercepted welding point images in the specified folder and perform template matching with the high-quality welding point images in the database to obtain the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images, and use the SIFT matching algorithm to judge the quality of the body-in-white welding points for the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images, and the specific content is as follows:

[0087] like Figure 8 As shown, the template graph ( Figure 8 Left) and real-time graph ( Figure 8 Right) establishes a set of key point descriptors (only one in the template point set and one in the real-time point set are shown in the figure). The matching and recognition of the target is completed by comparing the key point descriptors in the two point sets. The similarity measurement of the key point descriptors with 128 dimensions uses the Euclidean distance. Using the SIFT matching algorithm, as shown in formula (7), the ratio of the nearest neighbor distance to the next nearest neighbor distance from all points in the real-time point set to a certain point in the template image point set is calculated.

[0088]

[0089] Among them: D is the nearest neighbor distance, and d is the next nearest neighbor distance.

[0090] Generally, the best matching effect is achieved when the ratio is between 0.4 and 0.6. The threshold value T of the ratio of the nearest neighbor distance to the next nearest neighbor distance set in this patent is 0.4. If the ratio calculated above is less than the threshold, the pair of matching points will be retained. If it exceeds the threshold, the pair of matching points will be discarded. The smaller the threshold is, the fewer matching points will be retained, and the higher the matching degree with the solder joint template image in the database, indicating that the quality of the solder joint is better; otherwise, the matching effect is unstable and the quality of the solder joint is poor. According to this method, accurate judgment of the quality of the solder joint can be achieved.

[0091] Embodiment 2

[0092] Fig. 9 1 is a schematic structural block diagram of a body-in-white spot welding quality inspection device according to an exemplary embodiment, the device comprising:

[0093] An acquisition module 210 is used to acquire the welding point images collected by the camera, and pre-process the welding point images to obtain a plurality of processed welding point images;

[0094] A marking module 220, used for importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image;

[0095] A capture module 230, configured to capture the marked solder joint image to obtain a plurality of captured solder joint images and save them in a designated folder;

[0096] The judging module 240 is used to traverse the multiple intercepted welding point images in the specified folder and perform template matching with the high-quality welding point images in the database to obtain the corresponding scale factors of multiple feature points of the body-in-white welding points in the multiple images, and use the SIFT matching algorithm to judge the quality of the body-in-white welding points on the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images.

[0097] Preferably, the marking module 220 is used to:

[0098] The processed solder joint image is imported into the optimal YOLOv8s-FAC network model, the dilation rate is adjusted according to the image to capture the image details, the convolution kernel size is adaptively adjusted according to the image content, the feature information of different frequencies is balanced in the image, and the size of the feature map is gradually reduced and the number of channels is increased to obtain the high-level feature information of the image;

[0099] Inputting high-level feature information of the image into the head network of the YOLOv8s-FAC network, generating original predictions for target detection of the image through three detection heads, converting the original predictions of target detection of the image into coordinates and sizes of bounding boxes through feature maps output by the network, and converting probability density distribution into class probabilities to obtain solder joint images with multiple bounding box coordinates, class probabilities and confidence scores;

[0100] Traversing the solder joint images with multiple bounding box coordinates, category probabilities and confidence scores, filtering out bounding boxes with confidence scores lower than a confidence threshold, and obtaining solder joint images with multiple final bounding box coordinates, category labels and confidence scores, wherein the solder joint images with multiple final bounding box coordinates, category labels and confidence scores are multiple marked solder joint images.

[0101] The present application can quickly detect solder joint quality and can perform pixel-level detection and measurement, thereby avoiding human subjective errors and having higher detection accuracy and higher detection efficiency.

[0102] Embodiment 3

[0103] Fig.10 It is a structural block diagram of a terminal provided in an embodiment of the present application, and the terminal may be the terminal in the above embodiment.

[0104] Typically, the terminal 300 includes a processor 301 and a memory 302 .

[0105] The processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 301 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0106] The memory 302 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 302 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 302 is used to store at least one instruction, which is used to be executed by the processor 301 to implement a body-in-white spot welding quality detection method provided in this application.

[0107] In some embodiments, the terminal 300 may further include: a peripheral device interface 303 and at least one peripheral device. Specifically, the peripheral device includes: at least one of a radio frequency circuit 304 , a touch screen 305 , a camera 306 , an audio circuit 307 , a positioning component 308 and a power supply 309 .

[0108] The peripheral device interface 303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the peripheral device interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the peripheral device interface 303 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0109] The radio frequency circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 304 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 304 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 304 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 304 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0110] The touch display screen 305 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. The touch display screen 305 also has the ability to collect touch signals on the surface or above the surface of the touch display screen 305. The touch signal can be input to the processor 301 as a control signal for processing. The touch display screen 305 is used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the touch display screen 305 can be one, and the front panel of the terminal 300 is set; in other embodiments, the touch display screen 305 can be at least two, which are respectively set on different surfaces of the terminal 300 or are folded; in some other embodiments, the touch display screen 305 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 300. Even, the touch display screen 305 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The touch display screen 305 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0111] The camera assembly 306 is used to capture images or videos. Optionally, the camera assembly 306 includes a front camera and a rear camera. Typically, the front camera is used to realize video calls or selfies, and the rear camera is used to realize the shooting of photos or videos. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, and a wide-angle camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, and the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting functions. In some embodiments, the camera assembly 306 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0112] The audio circuit 307 is used to provide an audio interface between the user and the terminal 300. The audio circuit 307 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 301 for processing, or input them into the radio frequency circuit 304 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 300. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 301 or the radio frequency circuit 304 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 307 may also include a headphone jack.

[0113] The positioning component 308 is used to locate the current geographical location of the terminal 300 to implement navigation or LBS (Location Based Service). The positioning component 308 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0114] The power supply 309 is used to power various components in the terminal 300. The power supply 309 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 309 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0115] In some embodiments, the terminal 300 further includes one or more sensors 310 , including but not limited to: an acceleration sensor 311 , a gyroscope sensor 312 , a pressure sensor 313 , a fingerprint sensor 314 , an optical sensor 315 , and a proximity sensor 316 .

[0116] The acceleration sensor 311 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 300. For example, the acceleration sensor 311 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 301 can control the touch display screen 305 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 311. The acceleration sensor 311 can also be used for collecting game or user motion data.

[0117] The gyro sensor 312 can detect the body direction and rotation angle of the terminal 300. The gyro sensor 312 can cooperate with the acceleration sensor 311 to collect the user's 3D (3 Dimensions) action on the terminal 300. The processor 301 can implement the following functions based on the data collected by the gyro sensor 312: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0118] The pressure sensor 313 can be set on the side frame of the terminal 300 and / or the lower layer of the touch display screen 305. When the pressure sensor 313 is set on the side frame of the terminal 300, it can detect the user's holding signal of the terminal 300, and perform left and right hand recognition or shortcut operation according to the holding signal. When the pressure sensor 313 is set on the lower layer of the touch display screen 305, it can control the operability control on the UI interface according to the user's pressure operation on the touch display screen 305. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0119] The fingerprint sensor 314 is used to collect the user's fingerprint to identify the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 301 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 314 can be set on the front, back, or side of the terminal 300. When a physical button or a manufacturer logo is set on the terminal 300, the fingerprint sensor 314 can be integrated with the physical button or the manufacturer logo.

[0120] The optical sensor 315 is used to collect the ambient light intensity. In one embodiment, the processor 301 can control the display brightness of the touch display screen 305 according to the ambient light intensity collected by the optical sensor 315. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 305 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 305 is reduced. In another embodiment, the processor 301 can also dynamically adjust the shooting parameters of the camera assembly 306 according to the ambient light intensity collected by the optical sensor 315.

[0121] The proximity sensor 316, also called a distance sensor, is usually arranged on the front of the terminal 300. The proximity sensor 316 is used to collect the distance between the user and the front of the terminal 300. In one embodiment, when the proximity sensor 316 detects that the distance between the user and the front of the terminal 300 is gradually decreasing, the processor 301 controls the touch display screen 305 to switch from the screen-on state to the screen-off state; when the proximity sensor 316 detects that the distance between the user and the front of the terminal 300 is gradually increasing, the processor 301 controls the touch display screen 305 to switch from the screen-off state to the screen-on state.

[0122] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the terminal 300, and the terminal 300 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0123] Embodiment 4

[0124] In an exemplary embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the program is executed by a processor, a method for detecting the quality of spot welding of a body in white as provided in all the inventive embodiments of the present application is implemented.

[0125] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0126] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0127] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0128] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] Embodiment 5

[0130] In an exemplary embodiment, an application product is also provided, including one or more instructions, and the one or more instructions can be executed by the processor 301 of the above-mentioned device to complete the above-mentioned body-in-white spot welding quality detection method.

[0131] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A method for detecting the quality of spot welding of a body in white, characterized in that: include: Acquire the welding spot images captured by the camera, and pre-process the welding spot images to obtain a plurality of processed welding spot images; Importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image; The marked welding spot images are intercepted to obtain a plurality of intercepted welding spot images and saved in a designated folder; The multiple intercepted welding point images in the designated folder are traversed and template matching is performed with the high-quality welding point images in the database to obtain the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images, and the SIFT matching algorithm is used to judge the quality of the body-in-white welding points for the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images.

2. A body-in-white spot welding quality inspection method according to claim 1, characterized in that: Before obtaining the welding spot image captured by the camera, it also includes: obtaining a hand-eye calibration matrix, and controlling the camera at the end of the manipulator until the welding spot is accurately aligned to the center of the camera's field of view.

3. A body-in-white spot welding quality inspection method according to claim 1, characterized in that: Before obtaining the hand-eye calibration matrix, the method further includes: Collecting a plurality of high-quality body-in-white weld spot image data, processing and annotating the plurality of high-quality body-in-white weld spot image data to obtain a high-quality body-in-white weld spot image data set; A YOLOv8s-FAC network model for detecting the quality of body-in-white welds is constructed, and the YOLOv8s-FAC network model is trained using the high-quality body-in-white weld image dataset to obtain the optimal YOLOv8s-FAC network model.

4. A method for detecting the quality of spot welding of a body in white according to claim 1, characterized in that: The preprocessing of the solder joint image to obtain a processed solder joint image includes: Bilateral filtering is used to reduce noise in the solder joint image to remove noise information from the image, and the image is converted into the HSV color space. Feature enhancement is performed through the histogram, and data enhancement is performed on the image after feature enhancement. The image size is adjusted to the expected input size of the optimal YOLOv8s-FAC network model to obtain the processed solder joint image.

5. A method for detecting the quality of spot welding of a body in white according to claim 1, characterized in that: The step of importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image includes: The processed solder joint image is imported into the optimal YOLOv8s-FAC network model, the dilation rate is adjusted according to the image to capture the image details, the convolution kernel size is adaptively adjusted according to the image content, the feature information of different frequencies is balanced in the image, and the size of the feature map is gradually reduced and the number of channels is increased to obtain the high-level feature information of the image; Inputting high-level feature information of the image into the head network of the YOLOv8s-FAC network, generating original predictions for target detection of the image through three detection heads, converting the original predictions of target detection of the image into coordinates and sizes of bounding boxes through feature maps output by the network, and converting probability density distribution into class probabilities to obtain solder joint images with multiple bounding box coordinates, class probabilities and confidence scores; Traversing the solder joint images with multiple bounding box coordinates, category probabilities and confidence scores, filtering out bounding boxes with confidence scores lower than a confidence threshold, and obtaining solder joint images with multiple final bounding box coordinates, category labels and confidence scores, wherein the solder joint images with multiple final bounding box coordinates, category labels and confidence scores are multiple marked solder joint images.

6. A method for detecting the quality of spot welding of a body in white according to claim 5, characterized in that: The intercepting process of the marked welding spot image to obtain a plurality of intercepted welding spot images and saving them in a designated folder includes: The marked multiple welding spot images are intercepted by image clipping technology according to the final bounding box coordinates of each bounding box to obtain the intercepted multiple welding spot images; The captured multiple welding spot images are saved in a designated folder.

7. A body-in-white spot welding quality inspection device, characterized in that: include: An acquisition module is used to acquire the welding point images collected by the camera, and pre-process the welding point images to obtain a plurality of processed welding point images; A marking module, used for importing the processed solder joint image into the optimal YOLOv8s-FAC network model to obtain a marked solder joint image; A capture module, used for performing capture processing on the marked welding spot image to obtain a plurality of captured welding spot images and save them into a designated folder; The judging module is used to traverse the multiple intercepted welding point images in the specified folder and perform template matching with the high-quality welding point images in the database to obtain the corresponding scale factors of multiple feature points of the body-in-white welding points in the multiple images, and use the SIFT matching algorithm to judge the quality of the body-in-white welding points on the scale factors corresponding to the multiple feature points of the body-in-white welding points in the multiple images.

8. A body-in-white spot welding quality inspection device according to claim 7, characterized in that: The marking module is used for: The processed solder joint image is imported into the optimal YOLOv8s-FAC network model, the dilation rate is adjusted according to the image to capture the image details, the convolution kernel size is adaptively adjusted according to the image content, the feature information of different frequencies is balanced in the image, and the size of the feature map is gradually reduced and the number of channels is increased to obtain the high-level feature information of the image; Inputting high-level feature information of the image into the head network of the YOLOv8s-FAC network, generating original predictions for target detection of the image through three detection heads, converting the original predictions of target detection of the image into coordinates and sizes of bounding boxes through feature maps output by the network, and converting probability density distribution into class probabilities to obtain solder joint images with multiple bounding box coordinates, class probabilities and confidence scores; Traversing the solder joint images with multiple bounding box coordinates, category probabilities and confidence scores, filtering out bounding boxes with confidence scores lower than a confidence threshold, and obtaining solder joint images with multiple final bounding box coordinates, category labels and confidence scores, wherein the solder joint images with multiple final bounding box coordinates, category labels and confidence scores are multiple marked solder joint images.

9. A terminal, characterized in that: include: one or more processors; a memory for storing the one or more processor-executable instructions; Wherein, the one or more processors are configured to: Execute a body-in-white spot welding quality detection method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the body-in-white spot welding quality detection method as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Car body-in-white welding spot positioning method and device based on robot visual servo

    CN104942496A

  • Body-in-white welding spot 3D calibration method and system based on template matching and medium

    CN111830060A

  • Vehicle body welding spot detection method based on visual and ultrasonic detection

    CN116380890A

  • Template matching welding spot detection method in combination with Faster RCNN

    CN118247255A

Cited By

  • Automobile part production monitoring method and system

    CN119609291A

  • A method and system for monitoring the production of automotive parts

    CN119609291B

  • Automatic grinding method for automobile welding based on PLC signal triggering and related equipment

    CN121061704A