Vehicle body welding spot quality image detection method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of image processing, and discloses a vehicle body welding spot quality image detection method and device, computer equipment and a storage medium. According to the technical scheme, at least one image detection device is arranged in each preset area of the vehicle body to collect the welding spot image of the vehicle body, the welding spot sub-image corresponding to each welding spot is intercepted from the welding spot image, and then each welding spot sub-image is detected according to the preset detection algorithm; and the detection result of each welding spot sub-image is obtained, so that each welding spot can be detected, and the comprehensive detection of the welding spot quality is further realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method, device, computer device and storage medium for detecting the quality of vehicle body welding points by images. Background Art

[0002] In the process of automobile manufacturing, welding is one of the key steps in vehicle body assembly, directly affecting the structural strength and safety performance of the vehicle body. Whether the welding quality is good or not has an important impact on the overall quality of the vehicle. Therefore, it is crucial to detect the quality of automobile body welding points.

[0003] However, the existing technology uses manual inspection and sampling inspection, resulting in a low detection coverage rate and the problem that all abnormal welding points cannot be found in time. Summary of the Invention

[0004] In view of the above problems, this application provides a method, device, computer device and storage medium for detecting the quality of vehicle body welding points by images, which is used to solve the problem in the existing technology that the detection coverage rate is low and all abnormal welding points cannot be found in time, and realizes full coverage detection of each welding point to find abnormal welding points in time.

[0005] According to one aspect of the embodiments of this application, a method for detecting the quality of vehicle body welding points by images is provided. At least one image detection device is set in each preset area of the vehicle body. The method includes: collecting a welding point image of the vehicle body by using the image detection device; wherein, each welding point image includes at least one welding point; according to the position coordinates of the welding point in the welding point image, extracting the sub-images corresponding to the respective welding points from the welding point image; detecting each of the sub-images of the welding points according to a preset detection algorithm to obtain the detection results of each of the sub-images of the welding points; wherein, the detection results include normal welding points and abnormal welding points.

[0006] In an optional manner, the step of extracting the sub-images corresponding to the respective welding points from the welding point image according to the position coordinates of the welding point in the welding point image further includes: performing coordinate processing on the welding point image to obtain a coordinate-processed welding point image; identifying the welding points in the coordinate-processed welding point image to obtain the center point coordinates of the welding points and the coordinates of the smallest area where the welding points are located; according to the center point coordinates and the coordinates of the smallest area, intercepting the welding point image, and using the intercepted image as the sub-image corresponding to the welding point.

[0007] In an alternative manner, the step of detecting each of the solder joint sub-images according to a preset detection algorithm to obtain the detection results of each of the solder joint sub-images further includes: preprocessing the solder joint sub-image to obtain the target features of the solder joint sub-image; inputting the target features into the preset encoding model, and obtaining a reconstructed image sample output by the preset encoding model; wherein the preset encoding model is used for image reconstruction of the solder joint sub-image; respectively calculating the error value between the reconstructed image sample and a preset image sample; wherein the solder joint sub-image corresponding to the preset image sample is a normal solder joint; matching the error value with a preset threshold range, and obtaining the detection result of the solder joint sub-image according to the matching result.

[0008] In an alternative manner, the step of preprocessing the solder joint sub-image to obtain the target features of the solder joint sub-image further includes: performing color conversion on the solder joint sub-image to obtain a to-be-processed sub-image after color conversion; extracting the three-color brightness values of each pixel point in the to-be-processed sub-image, and forming a three-color brightness matrix; performing convolution calculation on the three-color brightness matrix by using a convolution kernel to obtain the target features corresponding to the solder joint sub-image.

[0009] In an alternative manner, before the step of inputting the target features into the preset encoding model and obtaining the reconstructed image sample output by the preset encoding model, it further includes: establishing an initial encoding model; wherein the input of the initial encoding model is the target features, and the output of the initial encoding model is the reconstructed image sample; training and testing the initial encoding model based on a preset training set and a preset test set respectively to obtain a preset encoding model; wherein the preset training set and the preset test set are established based on historical solder joint sub-images.

[0010] In an alternative manner, after the step of detecting each of the solder joint sub-images according to a preset detection algorithm to obtain the detection results of each of the solder joint sub-images, it further includes: taking the solder joint corresponding to the solder joint sub-image with an abnormal detection result as a target solder joint; taking the solder joint image where the target solder joint is located as a to-be-processed solder joint image, and obtaining the preset area where the corresponding image acquisition device is located as a target preset area; marking the target solder joint in the to-be-processed solder joint image according to the center point coordinates and / or the minimum area coordinates of the target solder joint to obtain a target solder joint image; transmitting the target preset area and the target solder joint image to a display interface for display.

[0011] In an alternative manner, after the step of marking the target solder joint in the solder joint image to be processed according to the center point coordinates of the target solder joint and / or the coordinates of the smallest area where the target solder joint is located to obtain the target solder joint image, the method further includes: establishing monitoring information about the target solder joint according to the target preset area and the target solder joint image; and sending the monitoring information to the Internet of Things platform so that the Internet of Things platform sends the monitoring information to relevant personnel.

[0012] According to another aspect of the embodiments of the present application, there is provided a vehicle body solder joint quality image detection device, including: an image acquisition module, configured to acquire a solder joint image of the vehicle body by using an image detection device; wherein, each solder joint image includes at least one solder joint; an image cropping module, configured to crop out the sub-solder joint images corresponding to the respective solder joints from the solder joint image according to the position coordinates of the solder joints in the solder joint image; and an image detection module, configured to detect each sub-solder joint image according to a preset detection algorithm to obtain a detection result of each sub-solder joint image; wherein, the detection result includes a normal solder joint and an abnormal solder joint.

[0013] According to another aspect of the embodiments of the present application, there is provided a computer device, including: a controller; and a memory, configured to store one or more programs, which, when executed by the controller, cause the controller to implement the vehicle body solder joint quality image detection method described in any one of the above-mentioned claims.

[0014] According to still another aspect of the embodiments of the present application, there is provided a computer-readable storage medium, in which at least one executable instruction is stored, and when the executable instruction runs on a computer device / equipment, it causes the computer device / equipment to perform the operations of the vehicle body solder joint quality image detection method described in any one of the above-mentioned claims.

[0015] In the embodiments of the present application, at least one image detection device is arranged in each preset area of the vehicle body to acquire a solder joint image of the vehicle body, and the sub-solder joint images corresponding to the respective solder joints are cropped out from the solder joint image, and then each sub-solder joint image is detected according to a preset detection algorithm to obtain a detection result of each sub-solder joint image, so that the detection of each solder joint can be realized, and further the comprehensive detection of the solder joint quality can be realized.

[0016] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to be able to understand the technical means of the embodiments of the present application more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and understandable, the following specifically describes the embodiments of the present application. Description of the Drawings

[0017] The accompanying drawings are only used to illustrate the embodiments and are not considered as a limitation to this application. Moreover, throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings:

[0018] Figure 1 A schematic flowchart of an embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0019] Figure 2 A schematic diagram of the installation positions of the light source holder and the camera in the image detection device in an embodiment is shown;

[0020] Figure 3 A solder joint image of the lower area of the B-pillar of the right front door frame of the vehicle body in an embodiment is shown;

[0021] Figure 4 It shows a solder joint sub-image cropped from Figure 3 in an embodiment;

[0022] Figure 5 A schematic flowchart of another embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0023] Figure 6 A schematic flowchart of yet another embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0024] Figure 7 A schematic flowchart of still another embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0025] Figure 8 A schematic flowchart of another embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0026] Figure 9 A schematic flowchart of yet another embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0027] Figure 10 A schematic diagram of the detection result display of the target solder joint in an embodiment is shown;

[0028] Figure 11 A schematic flowchart of still another embodiment of the vehicle body solder joint quality image detection method provided by this application is shown;

[0029] Figure 12 A schematic structural diagram of an embodiment of the vehicle body solder joint quality image detection device provided by this application is shown;

[0030] Figure 13 A schematic structural diagram of an embodiment of the computer device provided by this application is shown. Detailed implementation manners

[0031] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0032] The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the drawings are merely exemplary illustrations and do not necessarily include all content and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0034] As used in this application, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0035] In the process of automobile manufacturing, welding is one of the key steps in body assembly, directly affecting the structural strength and safety performance of the body. The quality of welding has an important impact on the overall quality of the vehicle. Therefore, it is crucial to detect the quality of the welding points on the automobile body.

[0036] Currently, the welding quality of the body-in-white of major automobile OEMs is usually detected by ultrasonic, semi-destructive, and destructive offline + manual methods, which are inefficient, have low coverage, and cannot detect problems in a timely manner, easily resulting in the potential risk of a large number of unqualified solder joints flowing out. Aiming at the problems of low efficiency of manual detection, low sampling coverage, and inability to detect and process abnormal solder joints in the existing technology, an image detection method for the quality of automobile body solder joints is proposed. This method is applied to the on-line monitoring system for spot welding quality. Through the real-time collection and detection of solder joint data, the detection efficiency and coverage of solder joints are greatly improved, and the comprehensive monitoring of solder joint quality is realized. For solder joints with abnormal detection, the system will automatically push the information of abnormal solder joints to the staff, facilitating the staff to directly locate the specific vehicle body number and solder joint number, and realizing the real-time control of abnormal solder joints. By adopting intelligent and digital means, the present invention improves the quality and efficiency of the welding process, realizes on-line high-precision real-time quality control, and is of great significance for improving welding quality, body strength, reducing costs, and improving production efficiency.

[0037] In order to better illustrate the technical solutions involved in the present invention, the following embodiments are used for detailed description:

[0038] Figure 1 The flowchart of an embodiment of the image detection method for the quality of the vehicle body solder joints of the present application is shown, and this method is executed by a computer device. It should be noted that in this embodiment, at least one image detection device needs to be set in each preset area of the vehicle body on the basis of the method implementation. As Figure 2 shown, the setting positions of the image detection devices required to implement the method in this embodiment are presented. The image detection device includes a camera and a light source. The placement positions of the light source and the camera directly affect the quality of the solder joint imaging. Figure 2 Then, a relatively better placement position diagram of the light source bracket and the camera is shown. The common preset areas of the vehicle body include the right front area of the roof, the A-pillar area, the lower area of the B-pillar, etc. In order to more completely collect the solder joints in each preset area of the vehicle body, at least one image detection device needs to be set in each preset area of the vehicle body to achieve full coverage of the solder joints.

[0039] Please refer to Figure 1 shown, this method includes the following steps:

[0040] Step S110: Use an image detection device to collect the solder joint images of the vehicle body.

[0041] Among them, each solder joint image includes at least one solder joint.

[0042] Specifically, since the image detection device collects the solder joint images of the vehicle body in a preset area, each solder joint image includes at least one solder joint, and usually multiple solder joints, such as Figure 3As shown, it presents the solder joint images in the lower area of the B-pillar of the right front door frame of the vehicle body, including 5 solder joints.

[0043] Step S120: According to the position coordinates of the solder joints in the solder joint image, extract the sub-images of the solder joints corresponding to each solder joint from the solder joint image.

[0044] Specifically, since each solder joint needs to be analyzed separately, it is necessary to extract each solder joint from the solder joint image to obtain the sub-images of the solder joints including a single solder joint. As Figure 4 shown, it is the sub-image of a solder joint extracted from Figure 3 which has one and only one solder joint.

[0045] Step S130: Detect each sub-image of the solder joint according to the preset detection algorithm to obtain the detection results of each sub-image of the solder joint.

[0046] Among them, the detection results include normal solder joints and abnormal solder joints.

[0047] Specifically, the preset detection algorithm is implemented based on a preset coding model. First, preprocess the sub-image of the solder joint to obtain the corresponding target features, and then input the target features into the preset coding model to obtain the reconstructed image sample of the output. Then, compare the difference between the reconstructed image sample and the preset image sample, calculate the error value between the two, and finally match the error value with the preset threshold interval to determine the detection result of the sub-image of the solder joint. The preset image sample is the image sample generated by the target features of the sub-image of the normal solder joint. Therefore, if the error value matches the preset threshold interval, it indicates that the detection result of the sub-image of the solder joint is a normal solder joint; if not, the detection result is an abnormal solder joint.

[0048] Beneficial effects: In this embodiment, at least one image detection device is set in each preset area of the vehicle body to collect the solder joint images of the vehicle body, extract the sub-images of the solder joints corresponding to each solder joint from the solder joint images, and then detect each sub-image of the solder joint according to the preset detection algorithm to obtain the detection results of each sub-image of the solder joint, so as to realize the detection of each solder joint and further realize the comprehensive detection of the solder joint quality.

[0049] By adopting advanced image processing algorithms and deep learning technologies, the system can immediately identify abnormal solder joints and trigger the abnormal alarm mechanism. This immediate feedback mechanism ensures that the welding personnel can immediately receive the abnormal alarm information, thus minimizing the time interval from the occurrence of the problem to its solution, realizing the real-time monitoring and immediate management of the solder joint quality. Based on this method, it can not only meet the high production rhythm of modern automobile manufacturing workshops, but also significantly improve the detection efficiency, effectively reduce the dependence on manual detection and the cost burden brought by it, ensure the real-time monitoring and timely correction of the welding quality, and greatly improve the product quality and production line stability.

[0050] In some embodiments, as Figure 5 shown, step S120 further includes:

[0051] Step S121: Perform coordinate processing on the solder joint image to obtain a coordinate-processed solder joint image.

[0052] Specifically, performing coordinate processing on the solder joint image means establishing a two-dimensional coordinate system based on the solder joint image to obtain a coordinate-processed solder joint image, such that each point in the solder joint image can be clearly represented by two-dimensional coordinates. This facilitates determining the coordinate positions of each solder joint in the solder joint image.

[0053] Step S122: Identify the solder joints in the coordinate-processed solder joint image to obtain the central point coordinates of the solder joints and the coordinates of the smallest region where the solder joints are located.

[0054] Specifically, taking the solder joint image in the Figure 3 right front door frame B-pillar lower region shown as an example, it is necessary to establish a two-dimensional coordinate system based on the solder joint image, and then identify the solder joints in the coordinate-processed solder joint image, that is, Figure 3 the solder joints are marked by the green marked frames in Figure 3 and the numbers on the green marked frames in

[0055] are the solder joint numbers. In real-time operation, a single solder joint sub-image can be directly named with the solder joint number, so that it can be detected one by one according to the solder joint number during detection to avoid omission. Figure 3 According to the coordinate-processed solder joint image, the central coordinates of the solder joints can be obtained, that is, the central coordinates (x, y) of each green marked frame in Figure 3 and the coordinates of the smallest region where the solder joints are located. The smallest region is the green marked frame in

[0056] The coordinates of the smallest region are determined according to the length and width (w, h) of the green marked frame and the central coordinates.

[0057] Specifically, taking Figure 3 as an example, according to the central point coordinates and the coordinates of the smallest region, the intercepted solder joint sub-image is the image of the green marked frame region in Figure 3 that is, the image of a single solder joint. The intercepted solder joint sub-image is as shown in Figure 4

[0058] ​Beneficial effects: In this embodiment, the steps of extracting the sub-images of each solder joint corresponding to the solder joints from the solder joint image according to the position coordinates of the solder joints in the solder joint image are further refined. By coordinate-transforming the solder joint image, it is possible to more quickly extract the corresponding sub-images of the solder joints according to the center point coordinates and minimum area coordinates of the solder joints, improving the speed and accuracy of identifying and extracting the sub-images of the solder joints based on the solder joint image.

[0059] In some embodiments, as Figure 6 shown, step S130 further includes:

[0060] Step S131: Preprocess the sub-image of the solder joint to obtain the target features of the sub-image of the solder joint.

[0061] Specifically, the preprocessing includes RGB value conversion and convolution operations to obtain the target features. The specific preprocessing will be described in detail in the next embodiment and will not be elaborated here.

[0062] Step S132: Input the target features into a preset encoding model and obtain the reconstructed image samples output by the preset encoding model.

[0063] Among them, the preset encoding model is used to reconstruct the image of the sub-image of the solder joint.

[0064] Specifically, using the variational autoencoder (VAE) model, sample reconstruction is performed on the target features to generate samples similar to the sub-images of the solder joints for subsequent reconstruction error calculation and anomaly detection. The VAE (variational autoencoder) generation model learns the probability distribution of the target features of the input sub-images of the solder joints and generates new, similar data samples. This technology is different from the existing supervised training learning. Supervised training learning requires a large amount of labeled data to improve the detection performance of the model. The VAE generation model is trained using unlabeled data, greatly saving the time for data labeling. The VAE generation model learns the data distribution of normal samples and establishes a distribution threshold, effectively solving the problem that there are few data samples of solder joint defects and it is impossible to establish a defect data set. In the case where there are few defect samples and it is impossible to construct a defect data set, the VAE (variational autoencoder) generation model has great advantages compared with supervised training learning.

[0065] Step S133: Calculate the error values between the reconstructed image samples and the preset image samples respectively.

[0066] Among them, the sub-image of the solder joint corresponding to the preset image sample is a normal solder joint. The preset image sample is the reconstructed sample output by inputting the target features extracted from the sub-image of the solder joint corresponding to the normal solder joint into the preset encoding model.

[0067] Specifically, an error value between a preset image sample and a reconstructed sample is calculated, and the error value reflects the abnormality degree of the image sample.

[0068] Step S134: matching the error value with a preset threshold interval, and obtaining a detection result of the solder joint sub-image according to the matching result.

[0069] Specifically, based on historical data (historical solder joint sub-image data), a threshold interval is automatically generated as a preset threshold interval for normal and abnormal judgment. For the solder joint sub-image to be detected, the corresponding error value is first calculated, and then the value is compared with the preset threshold interval. If the error value exceeds the threshold interval range, the image is detected as an abnormal solder joint. If the error value matches the preset threshold interval, the detection result corresponding to the solder joint sub-image is that the solder joint is normal.

[0070] Beneficial effects: This embodiment further refines the steps of detecting each solder joint sub-image according to a preset detection algorithm to obtain the detection results of each solder joint sub-image. The solder joint sub-image is preprocessed to extract the corresponding target features, and then the target features are input into the preset coding model, and the output reconstructed image samples are obtained, and then the error value between the reconstructed image sample and the preset image sample is calculated. Finally, according to the matching of the error value with the preset threshold range, the detection result of the corresponding solder joint sub-image is determined. Thus, simple mathematical calculations and numerical matching are used to replace complex image comparisons, thereby improving the detection rate and accuracy of each solder joint sub-image.

[0071] In some embodiments, Figure 7 As shown, step S131 further includes:

[0072] Step S210: performing color conversion on the solder joint sub-image to obtain a sub-image to be processed after color conversion.

[0073] Specifically, the captured welding spot image is converted from the BGR color space to RGB using the OpenCV library.

[0074] The RGB color space is defined by the chromaticity of the three primary colors red, green and blue, from which the corresponding color triangle can be defined to generate other colors. A complete RGB color space definition also requires the chromaticity of the white point and the gamma correction curve. BGR is the same as RGB, except that the order of the regions is reversed. Red occupies the least important region, green occupies the second place (stationary), and blue occupies the third place.

[0075] Step S220: extracting the three-color brightness value of each pixel in the sub-image to be processed, and forming a three-color brightness matrix.

[0076] Specifically, the RGB values of each pixel are obtained to form matrices for three channels of R, G, and B, preparing for subsequent feature extraction. The image has three two-dimensional matrices of R, G, and B, and the matrix values are between 0 and 255. The value size represents the amount of color allowed to pass through, which is also called the grayscale value. The larger the grayscale, the darker the corresponding color.

[0077] Step S230: Perform convolution calculation on the three-color brightness matrices using a convolution kernel to obtain the target features corresponding to the solder joint sub-images.

[0078] Specifically, perform convolution calculation on the matrices of the three channels of R, G, and B using a convolution kernel to extract the key features in the image, that is, the target features.

[0079] Beneficial effects: This embodiment further details the steps of preprocessing the solder joint sub-images to obtain the target features. Through color conversion, establishing three-color brightness matrices, and convolution calculation, the target features of the solder joint sub-images can be accurately obtained, enriching and perfecting the specific measures of preprocessing.

[0080] In some embodiments, as Figure 8 shown, before step S132, it further includes:

[0081] Step 310: Establish an initial coding model.

[0082] Among them, the input of the initial coding model is the target feature, and the output of the initial coding model is the reconstructed image sample.

[0083] Specifically, the initial coding model is generally a variational autoencoder (VAE) model. VAE is a generative model that generates similar samples by learning the distribution of normal data.

[0084] Step S320: Train and test the initial coding model based on a preset training set and a preset test set respectively to obtain a preset coding model.

[0085] Among them, the preset training set and the preset test set are established based on historical solder joint sub-images.

[0086] Specifically, the target features and the corresponding reconstructed image samples corresponding to each historical solder joint sub-image are divided into a preset training set and a preset test set according to a certain ratio. The target features in the preset training set are input into a machine learning model (i.e., the initial coding model) for training, and the weights of the model are continuously updated through an optimizer until the model weights reach the optimal state and can accurately construct the reconstructed image samples of the solder joint sub-images.

[0087] Beneficial effects: In this embodiment, by establishing an initial coding model and training the initial coding model to obtain a preset coding model, the modeling and model optimization are realized, so that the reconstructed image sample of the solder joint sub-image can be constructed more accurately, and the accuracy of the reconstructed image sample is improved.

[0088] In one embodiment, as Figure 9 shown, after step S130, it further includes:

[0089] Step S140: Take the solder joint corresponding to the solder joint sub-image with an abnormal detection result as the target solder joint.

[0090] Specifically, if the detection result is that the solder joint is abnormal, it means that the error value does not match the preset threshold. When the error value does not match the preset threshold, obtain the reconstructed image sample corresponding to the corresponding error value as the target reconstructed image sample, and obtain the target feature corresponding to the target reconstructed image sample and the solder joint sub-image from which the target feature is derived. Finally, the solder joint corresponding to the solder joint sub-image can be obtained, and this solder joint is used as the target solder joint.

[0091] Step S150: Take the solder joint image where the target solder joint is located as the solder joint image to be processed, and obtain the preset area where the corresponding image acquisition device is located as the target preset area.

[0092] Specifically, the solder joint image where the target solder joint is located is used as the solder joint image to be processed. Combining the above embodiments, the solder joint images are collected by different image acquisition devices, and different image acquisition devices are set in different preset areas of the vehicle body. According to the corresponding image acquisition device, the preset area of the vehicle body where the target solder joint is located can be known, and this preset area of the vehicle body is used as the target preset area.

[0093] Step S160: Mark the target solder joint in the solder joint image to be processed according to the center point coordinates and / or minimum area coordinates of the target solder joint to obtain the target solder joint image.

[0094] Specifically, according to the step of intercepting the solder joint sub-image from the solder joint image in the above embodiment, each solder joint corresponds to center point coordinates and minimum area coordinates in the solder joint image. The target solder joint can be marked in the solder joint image to be processed according to the center point coordinates or the minimum area coordinates. And marking the target solder joint from the solder joint image to be processed according to both the center point coordinates and the minimum area coordinates makes the marking of the target solder joint more accurate. After marking the target solder joint in the solder joint image to be processed, the target solder joint image with the target solder joint marked is obtained. As shown in the solder joint image in Figure 10 , the abnormal solder joint (i.e., the target solder joint) is marked with a red circle.

[0095] Step S170: Transmit the target preset area and the target solder joint image to the display interface for display.

[0096] Specifically, the specific display of the display interface is mainly for the convenience of relevant staff to directly see the abnormal solder joints. Therefore, the target preset area where the abnormal solder joint (target solder joint) is located and the target solder joint image must be displayed. The detection result will be displayed on the UI interface of the spot welding quality online monitoring system, specifically showing the abnormal area and the abnormal position, realizing the online monitoring of the solder joint quality, and facilitating the operator to track and determine the abnormal solder joint. In actual use, the display interface can be displayed as Figure 10 shown, which shows the preset area of the vehicle body. When relevant personnel click on the corresponding preset area, the target solder joint image of the corresponding preset area will be displayed, and the abnormal solder joints will also be prominently marked with red circles.

[0097] Beneficial effect: In this embodiment, by displaying and marking the target solder joints with abnormal detection results, the online monitoring of the solder joint quality is realized, which is convenient for the operator to track and determine the abnormal solder joints.

[0098] In one embodiment, as Figure 11 shown, after step S160, it further includes:

[0099] Step S410: Establish monitoring information about the target solder joint according to the target preset area and the target solder joint image.

[0100] Specifically, the monitoring information includes the body serial number BSN, the solder joint number, and the solder joint picture in the solder joint area, all of which are obtained according to the target preset area and the target solder joint image.

[0101] Step S420: Send the monitoring information to the Internet of Things platform so that the Internet of Things platform can send the monitoring information to relevant personnel.

[0102] Specifically, the target solder joint is the abnormal solder joint, and the monitoring information of the target solder joint is sent to the Internet of Things platform. The Internet of Things platform automatically pushes the abnormal information to the relevant responsible person. The abnormal information includes the body serial number BSN, the solder joint number, and the solder joint picture in the solder joint area, which is convenient for relevant personnel to directly locate the abnormal solder joint and realize the real-time control of the abnormal solder joint.

[0103] Beneficial effect: In this embodiment, according to the target preset area and the target solder joint image, monitoring information about the target solder joint is established, and the monitoring information is sent to relevant personnel through the Internet of Things platform, which is convenient for relevant personnel to directly locate the abnormal solder joint and realize the real-time control of the abnormal solder joint.

[0104] Figure 12 shows the structural schematic diagram of the embodiment of the vehicle body solder joint quality image detection device of the present application. Please refer to Figure 12As shown in the figure, the device 500 includes an image acquisition module 510, an image cropping module 520, and an image detection module 530, where:

[0105] The image acquisition module 510 is configured to acquire the solder joint images of the vehicle body by using an image detection device; wherein, each solder joint image includes at least one solder joint;

[0106] The image cropping module 520 is configured to crop out the sub-images corresponding to the respective solder joints from the solder joint images according to the position coordinates of the solder joints in the solder joint images;

[0107] The image detection module 530 is configured to detect each sub-image of the solder joint according to a preset detection algorithm to obtain the detection results of each sub-image of the solder joint; wherein, the detection results include normal solder joints and abnormal solder joints.

[0108] Advantageous effects: In this embodiment, at least one image detection device is arranged in each preset area of the vehicle body to acquire the solder joint images of the vehicle body, the sub-images corresponding to the respective solder joints are cropped out from the solder joint images, and then each sub-image of the solder joint is detected according to a preset detection algorithm to obtain the detection results of each sub-image of the solder joint, so as to be able to detect each solder joint, and further realize the comprehensive detection of the solder joint quality.

[0109] It should be noted that the vehicle body solder joint quality image detection device provided in the above embodiment and the vehicle body solder joint quality image detection method provided in the foregoing embodiment belong to the same concept. The specific manners in which each module and unit perform operations have been described in detail in the method embodiment, and will not be elaborated herein.

[0110] Figure 13 The structural schematic diagram of the embodiment of the computer device of the present application is shown, which shows the structural schematic diagram of the computer system of the computer device suitable for implementing the embodiment of the present application. The specific implementation of the computer device in the specific embodiment of the present application is not limited.

[0111] Please refer to Figure 13 As shown in the figure, the computer device includes: a controller; a memory for storing one or more programs, and when the one or more programs are executed by the controller, the vehicle body solder joint quality image detection method described above is executed.

[0112] Please continue to refer to Figure 13As shown, the computer system 600 of the computer device includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) 602 or the program loaded from the storage section 608 into the Random Access Memory (RAM) 503, such as executing the methods in the above embodiments. In the RAM 603, various programs and data required for system operation are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An Input / Output (I / O) interface 605 is also connected to the bus 604.

[0113] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including, for example, a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read from it can be installed into the storage section 608 as needed.

[0114] Specifically, according to the embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the Central Processing Unit (CPU) 601, various functions defined in the system of the present application are executed.

[0115] Another aspect of the present application also provides a computer-readable storage medium, in which at least one executable instruction is stored. When the executable instruction runs on a computer device / apparatus, it causes the computer device / apparatus to execute the vehicle body solder joint quality image detection method in any one of the above embodiments.

[0116] Beneficial effects: In the embodiment of the present application, at least one image detection device is arranged in each preset area of the vehicle body to collect the solder joint images of the vehicle body, and the sub-images corresponding to the respective solder joints are intercepted from the solder joint images, and then the preset detection algorithm is used to detect each sub-image of the solder joint to obtain the detection results of each sub-image of the solder joint, so that the detection of each solder joint can be realized, and further the comprehensive detection of the solder joint quality can be realized.

[0117] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable computer program is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The computer program included on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the above-mentioned module, segment of a program, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in an order different from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0119] The units involved in the embodiments described in the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the unit itself.

[0120] According to one aspect of the embodiments of the present application, a computer system is also provided, including a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage section into a random access memory (RAM), such as executing the methods in the above-mentioned embodiments. In the RAM, various programs and data required for system operation are also stored. The CPU, ROM, and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0121] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive as needed so that a computer program read therefrom is installed into the storage section as needed.

[0122] The above content is only a preferred exemplary embodiment of the present application and is not used to limit the implementation of the present application. Those of ordinary skill in the art can easily make corresponding adaptations or modifications according to the main concept and spirit of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope required by the claims.

Claims

1. A method for detecting the quality of body solder joints by image, characterized in that, At least one image detection device is provided in each preset area of the vehicle body, and the method includes: Using the image detection device to collect the solder joint images of the vehicle body; wherein, each of the solder joint images includes at least one solder joint; According to the position coordinates of the solder joints in the solder joint images, each solder joint sub-image corresponding to the solder joints is intercepted from the solder joint images; Detecting each of the solder joint sub-images according to a preset detection algorithm to obtain the detection results of each of the solder joint sub-images; wherein, the detection results include normal solder joints and abnormal solder joints.

2. The method according to claim 1, characterized in that, The step of intercepting each solder joint sub-image corresponding to the solder joints from the solder joint images according to the position coordinates of the solder joints in the solder joint images further includes: Performing coordinate transformation on the solder joint images to obtain coordinate-transformed solder joint images; Identifying the solder joints in the coordinate-transformed solder joint images to obtain the central point coordinates of the solder joints and the minimum area coordinates where the solder joints are located; According to the central point coordinates and the minimum area coordinates, intercept the solder joint images, and use the intercepted images as the solder joint sub-images corresponding to the solder joints.

3. The method according to claim 1, wherein The step of detecting each of the solder joint sub-images according to a preset detection algorithm to obtain the detection results of each of the solder joint sub-images further includes: Performing preprocessing on the solder joint sub-images to obtain the target features of the solder joint sub-images; Inputting the target features into the preset coding model and obtaining the reconstructed image samples output by the preset coding model; wherein, the preset coding model is used for image reconstruction of the solder joint sub-images; Calculating the error values between the reconstructed image samples and the preset image samples respectively; wherein, the solder joint sub-images corresponding to the preset image samples are normal solder joints; Matching the error values with a preset threshold interval, and obtaining the detection results of the solder joint sub-images according to the matching results.

4. The method according to claim 3, wherein The step of performing preprocessing on the solder joint sub-images to obtain the target features of the solder joint sub-images further includes: Performing color conversion on the solder joint sub-images to obtain the sub-images to be processed after color conversion; Extracting the three-color brightness values of each pixel point in the sub-images to be processed and forming a three-color brightness matrix; Performing convolution calculation on the three-color brightness matrix using a convolution kernel to obtain the target features corresponding to the solder joint sub-images.

5. The method according to claim 3, wherein Before the step of inputting the target features into the preset coding model and obtaining the reconstructed image samples output by the preset coding model, it further includes: Establishing an initial coding model; wherein, the input of the initial coding model is the target features, and the output of the initial coding model is the reconstructed image samples; Training and testing the initial coding model based on a preset training set and a preset test set respectively to obtain a preset coding model; wherein, the preset training set and the preset test set are established based on historical solder joint sub-images.

6. The method according to claim 2, characterized in that, After the step of detecting each of the solder joint sub-images according to a preset detection algorithm to obtain the detection results of each of the solder joint sub-images, it further includes: Regarding the solder joints corresponding to the solder joint sub-images with the detection result of abnormal solder joints as target solder joints; Take the solder joint image where the target solder joint is located as the solder joint image to be processed, and obtain the preset area where the corresponding image acquisition device is located as the target preset area; Mark the target solder joint in the solder joint image to be processed according to the center point coordinates and / or the minimum area coordinates of the target solder joint, so as to obtain the target solder joint image; Transmit the target preset area and the target solder joint image to the display interface for display.

7. The method according to claim 6, wherein After the step of marking the target solder joint in the solder joint image to be processed according to the center point coordinates and / or the minimum area coordinates of the target solder joint to obtain the target solder joint image, the method further includes: Establish monitoring information about the target solder joint according to the target preset area and the target solder joint image; Send the monitoring information to the Internet of Things platform so that the Internet of Things platform sends the monitoring information to relevant personnel.

8. An image detection device for the quality of body weld points, characterized in that, The device includes: An image acquisition module, configured to acquire the solder joint image of the vehicle body by using an image detection device; wherein, each solder joint image includes at least one solder joint; An image intercepting module, configured to intercept the sub-solder joint images corresponding to the solder joints from the solder joint image according to the position coordinates of the solder joints in the solder joint image; An image detection module, configured to detect each sub-solder joint image according to a preset detection algorithm to obtain the detection results of each sub-solder joint image; wherein, the detection results include normal solder joints and abnormal solder joints.

9. A computer device, characterized in that, It includes: A controller; A memory, configured to store one or more programs, when the one or more programs are executed by the controller, the controller implements the vehicle body solder joint quality image detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, At least one executable instruction is stored in the storage medium, and when the executable instruction runs on a computer device / equipment, the computer device / equipment executes the operations of the vehicle body solder joint quality image detection method according to any one of claims 1 to 7.

Images