A method and system for detecting defects in top cover welding based on multiple cameras

Abstract

The present invention discloses a method and system for detecting the quality of top cover welding defects based on multiple cameras. The present invention obtains the control data for starting the camera shooting through the method, and generates image data corresponding to the top cover welding defects and at different angles in real time according to the control data; analyzes and processes the image data corresponding to the top cover welding defects and at different angles, and then calculates and processes them to obtain the quality detection data corresponding to the top cover welding defects, as well as the system, platform and storage medium corresponding to the method; combines the principle of image imaging, detects the quality of the weld by a high-quality imaging method and a method combining 3D machine vision and deep learning, and obtains a clear actual image of the defect by improving the imaging quality. Combining the advantages of 2D machine vision, 3D machine vision and deep learning methods, the overkill and missed kill can be greatly reduced. That is, small pinholes, blackening and cracks can be successfully detected, and the risk of missed detection of such defects can be successfully solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect quality detection and processing, and in particular relates to a top cover welding defect quality detection method and system based on multiple cameras. Background Art

[0002] At present, in the production process of power batteries, the quality of the laser welding seams around the top cover is very important. If there are bulges, pits, and uneven welds in the welds, these defects will affect the installation of the battery module. During use, it is easy to be damaged and cause safety accidents. What is more serious is that it is difficult for the human eye to distinguish these defects. The product is very easy to flow into the next station, and these defects cannot be detected by the helium inspection of the next station. Therefore, in the production process, many factories strictly require 100% online automatic detection.

[0003] At present, most companies still use workers to achieve 100% manual inspection under a microscope camera. This inspection method is affected by the worker's proficiency, mood and fatigue. It is easy to miss some small defects such as pits, cracks and pinholes, especially when these defects occur on the inside of the weld, the outside of the weld or the R corner. If the imaging conditions of these parts are not enhanced by imaging, the 3D camera cannot clearly image the defect characteristics of these parts. Especially for those defects with high information, such as high or low welds, it is difficult to distinguish them with the human eye.

[0004] In this situation, new artificial intelligence detection technology that replaces manual detection has emerged. At present, the weld quality detection of the top cover of the aluminum shell of square box batteries mainly adopts 2D machine vision, 3D machine vision, deep learning and other methods to achieve weld quality detection. If only one of these detection methods is used in the entire detection system, there is a serious problem that the detection system will miss or over-kill, which makes it difficult to apply to large-scale generation needs.

[0005] That is to say, if the system only uses 2D machine vision, 2D machine vision identifies defects based on the gray value changes of the image when analyzing and processing defects. It cannot obtain the depth information of the product and cannot judge defects related to depth information, such as pits, bulges, and uneven weld height defects. This is a fatal shortcoming of 2D machine vision. If the system only uses 3D machine vision, 3D machine vision identifies defects based on depth information. For some defects with unclear depth information, such as burnt black, cracks, small pinholes, etc., 3D machine vision cannot detect them. For some defects that occur on the inside, outside or R angle of the weld, if the 3D camera collects point cloud data from directly above the weld, it is difficult for the 3D camera to obtain a complete defect image. In this case, neither 2D machine vision nor 3D machine vision can identify defects. Only by improving the imaging conditions can the defects be shown on the image. Some use the camera laser line to scan at a 45-degree angle with the weld. This method makes the 3D camera receiver scan two edges with the receiver on the inside of the weld, and scan the other two edges with the receiver on the outside of the weld. This results in clear point cloud data for two edges, and strong noise interference for the point cloud images of the other two edges. There are misjudgments in the R angle of the top cover weld, and defects occur frequently at the R angle, because this position is where the weld ends and the other side begins. The speed changes greatly during the welding process, which makes it easy to form defects in the weld, and this defect is difficult to be expressed by normal imaging methods. In the production process of power batteries, fast detection speed and high detection accuracy are required. These requirements make it impossible to detect some small defects in the weld. This problem has always been one of the difficulties in weld quality inspection, because the higher the image acquisition speed, the lower the resolution of the camera in the Y direction, and the information that characterizes the defect and the information that can be used to characterize the defect characteristics are relatively small. Traditional image processing methods are difficult to solve these problems. The results of automated inspection must be consistent with the results of manual eye inspection. Because the human eye is subjective when observing defects, different people will have different results for the same defect. In addition, the human eye can also look at the defect position from different angles and heights to obtain the overall characteristics of the defect. This requires our detection system to have a strong learning ability to meet these misjudgments caused by subjective assumptions. Due to reasons such as imaging quality and software defect detection methods, the detection results may have high rates of missed kills and over-kills.

[0006] Therefore, in view of the above technical problems and defects, it is urgent to design and develop a top cover welding defect quality detection method and system based on multiple cameras. Summary of the invention

[0007] In order to overcome the shortcomings and difficulties of the above-mentioned prior art, the purpose of the present invention is to provide a top cover welding defect quality inspection method, system, platform and storage medium based on multiple cameras, which combines the advantages of 2D machine vision, 3D machine vision and deep learning methods, and can greatly reduce overkill and prevent missed kills.

[0008] The first object of the present invention is to provide a method for detecting defects in top cover welding based on multiple cameras;

[0009] The second object of the present invention is to provide a top cover welding defect quality inspection system based on multiple cameras;

[0010] The third object of the present invention is to provide a top cover welding defect quality inspection platform based on multiple cameras;

[0011] A fourth object of the present invention is to provide a computer-readable storage medium;

[0012] The first object of the present invention is achieved in that the method comprises the following steps:

[0013] Acquire control data for starting camera shooting, and generate image data corresponding to the top cover welding defect and at different angles in real time according to the control data;

[0014] The image data corresponding to the top cover welding defect and at different angles are analyzed and processed, and quality inspection data corresponding to the top cover welding defect is generated.

[0015] Furthermore, the acquisition of control data for starting camera shooting and the generation of image data corresponding to the top cover welding defect and at different angles in real time according to the control data also include:

[0016] Image data corresponding to the top cover welding defect is acquired, and a detection mathematical model corresponding to the top cover welding defect is constructed according to the image data.

[0017] Further, the analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and generating quality inspection data corresponding to the top cover welding defect, also includes:

[0018] Acquire point cloud data of image data corresponding to the top cover welding defect, and convert the point cloud data into depth image data;

[0019] The edge segment data of the weld is obtained, and the defective image is corrected by rotating it in real time according to the edge segment data; that is, the angle between the weld and the horizontal direction is obtained, and the weld is rotated to a vertical position in the horizontal direction according to the edge data and the horizontal angle, and the correction is performed by rotating it in real time to obtain an image that is consistent with the standard image orientation, thereby achieving the purpose of image correction.

[0020] Generate weld area position data, and divide the weld area position data into area types in real time.

[0021] Furthermore, the step of acquiring point cloud data of image data corresponding to the top cover welding defect and converting the point cloud data into depth image data further includes:

[0022] According to the point cloud data of the image data, outliers in the point cloud data are removed. That is, according to the point cloud data of the product to be inspected, outliers in the point cloud data are removed.

[0023] Furthermore, the generating of the weld area position data and real-time classification of the weld area position data into area types also includes:

[0024] Divide and generate defect processing areas, and use convex hull algorithm to detect defect areas in real time;

[0025] Divide and generate the inner and outer defect detection areas of the weld, and perform image transformation processing on the depth image corresponding to the defect detection area;

[0026] The R-angle defect detection area is divided and generated, and the characteristic values ​​corresponding to the weld defects are generated by combining the profile contour method, where the characteristic values ​​include: bulges, pits, burst points, pinhole depth and length.

[0027] Furthermore, the division generates the inner and outer defect detection areas of the weld, and performs image transformation processing on the depth image corresponding to the defect detection area, and further includes:

[0028] Generate weld height position data, and generate weld inner position data and weld outer position data respectively according to the weld height position data;

[0029] The weld side wall point cloud data and cap vertex point cloud data are acquired and processed respectively.

[0030] The second object of the present invention is achieved in that the system comprises:

[0031] A data acquisition and generation unit, used to acquire control data for starting camera shooting, and to generate image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera;

[0032] The data analysis and processing generating unit is used to analyze and process the image data corresponding to the top cover welding defects and at different angles, and generate quality inspection data corresponding to the top cover welding defects.

[0033] Furthermore, the data acquisition generation unit further includes:

[0034] A model building module, used to obtain image data corresponding to the top cover welding defect, and build a detection mathematical model corresponding to the top cover welding defect according to the image data;

[0035] The data analysis processing generation unit further includes:

[0036] An acquisition conversion module is used to acquire point cloud data of image data corresponding to the top cover welding defect, and convert the point cloud data into depth image data;

[0037] The rotation correction module is used to obtain the edge segment data of the weld and to perform real-time rotation correction on the defective image based on the edge segment data; that is, to obtain the angle between the weld and the horizontal direction, and to rotate the weld to a vertical position in the horizontal direction based on the edge data and the horizontal angle, and to perform real-time rotation correction to obtain an image that is consistent with the standard image orientation, thereby achieving the purpose of image correction.

[0038] A generation and division module is used to generate weld area position data and divide the weld area position data into area types in real time;

[0039] And / or, obtaining a conversion module further includes:

[0040] A data removal module, used for removing outliers in the point cloud data according to the point cloud data of the image data;

[0041] And / or, generating a partitioning module further includes:

[0042] The first division generation module is used to divide and generate defect processing areas, and detect defect areas in real time in combination with a convex hull algorithm;

[0043] The second division generation module is used to divide and generate the inner and outer defect detection areas of the weld, and perform image transformation processing on the depth image corresponding to the defect detection area;

[0044] The third division generation module is used to divide and generate the R corner defect detection area, and generate the characteristic values ​​corresponding to the weld defects in combination with the profile contour method, wherein the characteristic values ​​include: bulges, pits, burst points, pinhole depth and length;

[0045] And / or, the second partition generation module further includes:

[0046] A generating module, used to generate weld height position data, and generate weld inner position data and weld outer position data according to the weld height position data;

[0047] The data processing module is used to obtain and process the weld side wall point cloud data and the cap vertex point cloud data respectively.

[0048] The third object of the present invention is achieved as follows: it includes a processor, a memory and a top cover weld defect quality detection platform control program based on multiple cameras; wherein the top cover weld defect quality detection platform control program based on multiple cameras is executed by the processor, and the top cover weld defect quality detection platform control program based on multiple cameras is stored in the memory, and the top cover weld defect quality detection platform control program based on multiple cameras implements the top cover weld defect quality detection method based on multiple cameras.

[0049] The fourth object of the present invention is achieved in this way: the computer-readable storage medium stores a multi-camera based top cover weld defect quality detection platform control program, and the multi-camera based top cover weld defect quality detection platform control program implements the multi-camera based top cover weld defect quality detection method.

[0050] The present invention obtains control data for starting camera shooting through a method, and generates image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera; analyzes and processes the image data corresponding to the top cover welding defect and at different angles, and generates quality inspection data corresponding to the top cover welding defect, as well as a system, platform and storage medium corresponding to the method; combines the principle of image imaging, detects the quality of the weld by a high-quality imaging method and a method combining 3D machine vision and deep learning, and obtains a clear actual image of the defect by improving the imaging quality, and combines the advantages of 2D machine vision, 3D machine vision and deep learning methods to greatly reduce overkill and prevent missed kills. That is, small pinholes, blackening and cracks can be successfully detected, and the risk of missed detection of such defects can be successfully solved.

[0051] In other words, the outlier removal algorithm can successfully remove outlier information, prevent it from affecting subsequent processing, and improve the reliability of detection. The camera installation inclination angle and the movement direction inclination angle can effectively reduce interference signals and improve the accuracy and reliability of detection. The image correction function can effectively reduce overkill, and the algorithm can easily locate the weld, improving the accuracy of detection. The convex hull small defect detection function can easily and accurately detect small defects on the weld. Combined with the weld inside and outside defect algorithms, the defects on the inside and outside of the weld and the side wall can be accurately detected to prevent the detection system from missing. And through the R angle classification processing algorithm, the R angle can be well prevented from missing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is a schematic flow chart of a method for detecting defects in top cover welding based on multiple cameras of the present invention;

[0054] Figure 2 It is a schematic diagram of an embodiment of a method for detecting defects in top cover welding based on multiple cameras of the present invention;

[0055] Figure 3 This is a schematic diagram of the architecture of a roof welding defect quality inspection system based on multiple cameras of the present invention;

[0056] Figure 4 This is a schematic diagram of the architecture of a top cover welding defect quality inspection platform based on multiple cameras of the present invention;

[0057] Figure 5 A schematic diagram of a computer-readable storage medium architecture in an embodiment of the present invention;

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] In order to better understand the purpose, technical solutions and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0060] The present invention may also be implemented or applied through other different specific examples, and the details in this specification may also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0061] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0062] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. Secondly, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0063] Preferably, the multi-camera-based top cover welding defect quality detection method of the present invention is applied in one or more terminals or servers. The terminal is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0064] The terminal can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal can interact with the client through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

[0065] The present invention is to realize a method, system, platform and storage medium for detecting top cover welding defects based on multiple cameras.

[0066] like Figure 1 , which is a flow chart of a method for detecting top cover welding defects based on multiple cameras provided in an embodiment of the present invention.

[0067] In this embodiment, the top cover welding defect quality inspection method based on multiple cameras can be applied to a terminal with a display function or a fixed terminal. The terminal is not limited to a personal computer, a smart phone, a tablet computer, a desktop computer or an all-in-one computer equipped with a camera, etc.

[0068] The multi-camera-based roof welding defect quality detection method can also be applied to a hardware environment consisting of a terminal and a server connected to the terminal via a network. The network includes but is not limited to: a wide area network, a metropolitan area network, or a local area network. The multi-camera-based roof welding defect quality detection method of the embodiment of the present invention can be executed by a server, or by a terminal, or by both a server and a terminal.

[0069] For example, for a terminal that needs to perform multi-camera top cover welding defect quality inspection, the multi-camera top cover welding defect quality inspection function provided by the method of the present invention can be directly integrated on the terminal, or a client for implementing the method of the present invention can be installed. For another example, the method provided by the present invention can also be run on a server or other device in the form of a software development kit (SDK), and an interface for the multi-camera top cover welding defect quality inspection function is provided in the form of SDK. The terminal or other device can implement the multi-camera top cover welding defect quality inspection function through the provided interface. The present invention is further described below in conjunction with the accompanying drawings.

[0070] like Figure 1-Figure 2 As shown, the present invention provides a method for detecting top cover welding defects based on multiple cameras, and the method comprises the following steps:

[0071] S1, obtaining control data for starting camera shooting, and generating image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera;

[0072] S2, analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and generating quality inspection data corresponding to the top cover welding defect. That is, analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and then processing, analyzing and calculating to obtain quality inspection data corresponding to the top cover welding defect;

[0073] The step of acquiring control data for starting the camera to shoot, and generating image data corresponding to the top cover welding defect and at different angles in real time according to the control data, further comprises:

[0074] S11, acquiring image data corresponding to the top cover welding defect, and constructing a detection mathematical model corresponding to the top cover welding defect according to the image data.

[0075] The analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and generating quality inspection data corresponding to the top cover welding defect, also includes:

[0076] S21, obtaining point cloud data of image data corresponding to the top cover welding defect (such as obtaining weld data), and converting the point cloud data into depth image data;

[0077] S22, obtaining edge line segment data of the weld, and rotating and correcting the defective image in real time according to the edge line segment data; in other words, obtaining the edge coordinate value of the weld, and calculating the angle between the weld and the horizontal direction according to the edge coordinate value, and rotating and correcting the defective image in real time; that is, obtaining the angle between the weld and the horizontal direction, and rotating the weld to a vertical position in the horizontal direction according to the edge data and the horizontal angle, and rotating and correcting it in real time to obtain an image consistent with the standard image orientation, thereby achieving the purpose of image correction.

[0078] S23, generating weld area position data, and dividing the weld area position data into area types in real time.

[0079] The step of acquiring point cloud data of image data corresponding to the top cover welding defect and converting the point cloud data into depth image data further includes:

[0080] S211. According to the point cloud data of the image data, outliers in the point cloud data are removed.

[0081] The generating of the weld area position data and real-time classification of the weld area position data into area types also includes:

[0082] S231, dividing and generating a defect processing area, and combining a convex hull algorithm to detect the defect area in real time; or dividing and detecting defect positions;

[0083] S232, dividing and generating the inner and outer defect detection areas of the weld, and performing image transformation processing on the depth image corresponding to the defect detection area; that is, performing image processing on the depth image corresponding to the defect detection area;

[0084] S233. Divide and generate the R-angle defect detection area, and combine the profile contour method to generate the characteristic value corresponding to the weld defect, that is, combine the profile contour method to process, analyze and calculate to obtain the characteristic value corresponding to the weld defect; wherein the characteristic value includes: protrusion, pit, burst point, pinhole depth and length, which are specifically reflected in the weld height, weld width, weld lowest point and its length.

[0085] The division generates inner and outer defect detection areas of the weld, and performs image transformation processing on the depth image corresponding to the defect detection area, and further includes:

[0086] S2321, generating weld height position data (i.e., positioning height data), and generating weld inner position data and weld outer position data respectively according to the weld height position data (i.e., searching for weld inner position data and weld outer position data respectively);

[0087] S2322. Acquire and process the weld side wall point cloud data and the cap vertex point cloud data respectively.

[0088] Specifically, in the embodiment of the present invention, in the detection algorithm, for defects that can be directly identified by depth information, 3D machine vision technology is used to implement, such as broken welds, burst points, pits, protrusions, and pinholes; for defects where the depth information is not obvious, 2D machine vision can clearly obtain the defect feature information by using a deep learning method, through deep learning, to identify the approximate location of the defect, and then re-judge where the defect may appear, such as cracks, blackening, small pinholes, cold welding, etc. This re-judgment method is selected according to the characteristics of the defect, such as area, aspect ratio, depth information, etc., and the accuracy of the judgment is greatly improved through these re-judgment conditions. The purpose is to solve the risk of missing small defects in the system through a small defect detection algorithm; combined with defect mathematical modeling, the weld is searched through the key characteristic values ​​of the weld and the secondary judgment is made whether there is a defect. When searching for the key feature points of the weld, the interference of outliers is removed, and the result is closer to the true value.

[0089] This application proposes a weld quality inspection solution based on artificial intelligence 3D machine vision, which solves this difficulty by improving weld imaging conditions and software detection algorithms. Among them, the 3D camera imaging system improves the imaging quality of the image through the following measures. Generally, the 3D camera light-emitting surface moves parallel to the weld surface to scan the weld point cloud data. This method will cause the weld defects on the side wall to be unable to be imaged. We propose a method in which the 3D camera light-emitting surface is at a 30-degree angle to the weld plane to scan and image. This imaging method can clearly see the defects on the inside and outside of the weld. By keeping the 3D camera light-emitting surface at an inclination of 3 to 10 degrees with the direction of movement, when the laser encounters a bright weld, the stray light will be changed to produce diffuse reflection and will not return to the camera receiving end, thereby reducing the interference of the weld surface caused by the brightness of the weld surface and reducing the product overkill rate.

[0090] Camera installation angle compensation and height direction compensation. Collect the camera profile contour line, shine the laser line on the product surface, and calculate the angle between the profile contour line and the horizontal plane to get the camera installation angle. Compensation is performed according to the installation angle compensation formula. Generally, 3D cameras provide corresponding installation angle compensation and height direction compensation.

[0091] Select the appropriate height detection range and optimize the camera acquisition parameter design. Under the premise of meeting the Z direction field of view, try to reduce the Y direction resolution. In this way, the 3D camera can obtain higher precision images.

[0092] The entire testing system tests 2 products at a time, and its testing process is as follows:

[0093] The product is transferred to the fixture by the robot, and the 2D camera, driven by the module, starts to take images of battery cell 1 and sends the images to the computer for deep learning detection.

[0094] Long side 3D camera-1 captures images of long side 1 of battery cell 1, and long side 3D camera-2 captures images of long side 2 of battery cell 1.

[0095] Driven by the module, the camera starts to take images of battery cell 2 and sends the images to the computer for deep learning detection.

[0096] Long side 3D camera-1 captures images of long side 1 of cell 2, and long side 3D camera-2 captures images of long side 2 of cell 2.

[0097] The short side 3D camera-1 and the short side 3D camera-2 scan the battery cell-1 and the battery cell-2 simultaneously.

[0098] The test results are obtained through image processing, analysis and calculation, and the product results are judged in combination with AI results.

[0099] The software detection implementation process is as follows:

[0100] Before production testing, the following preparations need to be done:

[0101] (1) Prepare at least 50 images for each defect and 1,000 OK images for sample training and testing of deep learning and for subsequent production testing.

[0102] (2) Use the sample images above to perform 3D algorithm testing, perform mathematical modeling on various defects, and obtain their mathematical models for subsequent testing.

[0103] 3D algorithm realizes defect detection process

[0104] The following is the preprocessing process of 3D weld defect detection. The specific steps are as follows:

[0105] (1) The 3D camera collects point cloud data and uploads it to the industrial computer, where the point cloud data is converted into a depth image and displayed on the display.

[0106] (2) Remove outliers from the point cloud data to reduce their interference in the detection process. The process of removing outliers is as follows:

[0107] Convert 16-bit depth image to 8-bit image; Threshold extraction to distinguish objects from background; Morphological processing to separate or connect adjacent particles; Contour extraction to remove outliers; Regenerate 16-bit depth image;

[0108] (3) Weld seam correction can improve the accuracy of weld seam detection. Its implementation process is as follows:

[0109] In the weld ROI, search for the edge line segment line1 of the weld; calculate the angle between line1 and the horizontal line; calculate the rotation matrix of the line; perform rotation correction on the image; search for the weld line line1 again to obtain the rotated line line2.

[0110] (4) Weld pre-positioning: In order to accurately detect weld defects, the weld needs to be positioned. This can be done by searching for the top of the cell, the edge of the weld, and the line segment at the bottom of the cell, and then calculating the intersection of the top and the weld, and the intersection of the bottom and the weld. The line between the two intersections is the length of the weld. This can achieve the weld pre-positioning function.

[0111] (5) According to the weld position, the long side weld is divided into 5 areas and the short side weld is divided into 2 areas, and the reference height value of each area is generated.

[0112] (6) Regional small defect processing Small defects are detected in each small weld inspection area. In the process of small defect detection, the image processing algorithm is very important. Misjudgment is often caused by the interference of some small signals on the image. For small defects, we use the convex hull algorithm to detect small defects. The specific algorithm process is as follows:

[0113] Set a small defect detection area ROI in the weld, select a reference area within the ROI, and read the average height value of this area. Set the upper and lower limits of the extraction of the small defect detection area ROI, and extract the area according to the upper and lower limits. Use the Otsu method to perform threshold extraction on the area to obtain a binary image. Perform morphological processing on the binary image to remove small particles and merge the areas. Contour extraction, convex hull processing is performed on those that meet the contour conditions. Calculate the convex hull value and determine whether there are defects. The small defect detection results in each cell are the small defect detection results of the entire weld.

[0114] (7) Regional profile contour line search for defects In each divided area, the distribution of the profile contour line is calculated, processed and analyzed to determine whether the weld has defects. The entire judgment steps are as follows:

[0115] In each inspection area, find the outermost and innermost points of the weld, and search for the maximum and minimum values ​​of the weld height and weld pits between these two positions. Calculate the key characteristic values ​​of the weld, the lengths of bulges, pits, burst points and pinholes, and determine whether there may be defects based on the mathematical model of the defect. If there is a defect, search for the difference between the maximum and minimum values ​​of the area in the Y direction within the range of the possible defect locations as the basis for determining whether there is a defect. Combine the results of each inspection area as the inspection result of the weld edge.

[0116] (8) For small crack defects, deep learning methods are used to achieve this detection purpose from 2D images. This is mainly because the 3D camera has low resolution in the X and Y directions and these defects cannot be displayed in 3D images. Therefore, it is necessary to use 2D images and deep learning methods to achieve this detection purpose.

[0117] (9) Small defect detection algorithm on the inside and outside of the weld, which can accurately detect the defects on the inside and outside. Its implementation steps are as follows:

[0118] Perform image transformation on the depth image, taking the highest point of the weld as the center, divide the weld into two parts: the inner side and the outer side, and then process the side wall and cap vertex clouds separately. Rotate the cap vertex cloud and the side wall point cloud to the horizontal plane, and then perform depth extraction to remove redundant point cloud information. Convert the 16-bit depth image to an 8-bit grayscale image. Perform threshold processing and morphological processing. Perform contour extraction, extract key feature values, and then determine whether there are defects.

[0119] (10) R corner defect detection is the place where the long and short sides of the weld switch, and it is also the place where the speed changes fastest during the welding process. It is easy to form various defects. In order to solve this defect, the following steps can be taken:

[0120] Quickly locate the R angle position and classify the R angle position into the welding start R angle and welding end R angle. Use the profile contour method to calculate the key characteristic values ​​of the weld, such as the depth and length of the bulge, pit, explosion point, and pinhole. Detect and compare the two types of R angles with the standard value respectively. Those that meet the conditions are judged as defects, otherwise they are OK values. Generate the test results.

[0121] To achieve the above object, the present invention also provides a top cover welding defect quality inspection system based on multiple cameras, such as Figure 3 As shown, the system specifically includes:

[0122] A data acquisition and generation unit, used to acquire control data for starting camera shooting, and to generate image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera;

[0123] The data analysis and processing generating unit is used to analyze and process the image data corresponding to the top cover welding defect and at different angles, and generate quality inspection data corresponding to the top cover welding defect. That is, the image data corresponding to the top cover welding defect and at different angles is analyzed and processed, and then processed, analyzed and calculated to obtain quality inspection data corresponding to the top cover welding defect;

[0124] Furthermore, the data acquisition generation unit further includes:

[0125] A model building module, used to obtain image data corresponding to the top cover welding defect, and build a detection mathematical model corresponding to the top cover welding defect according to the image data;

[0126] The data analysis processing generation unit further includes:

[0127] An acquisition conversion module is used to acquire point cloud data of image data corresponding to the top cover welding defect (such as acquiring weld data), and convert the point cloud data into depth image data;

[0128] The acquisition rotation correction module is used to acquire the edge line segment data of the weld, and to rotate and correct the defective image in real time according to the edge line segment data; in other words, to acquire the edge coordinate value of the weld, and to calculate the angle between the weld and the horizontal direction according to the edge coordinate value, and to rotate and correct the defective image in real time; that is, to acquire the angle between the weld and the horizontal direction, and to rotate the weld to a vertical position in the horizontal direction according to the edge data and the horizontal angle, and to perform real-time rotation correction to obtain an image consistent with the standard image orientation, thereby achieving the purpose of image correction.

[0129] A generation and division module is used to generate weld area position data and divide the weld area position data into area types in real time;

[0130] And / or, obtaining a conversion module further includes:

[0131] A data removal module, used for removing outliers in the point cloud data according to the point cloud data of the image data;

[0132] And / or, generating a partitioning module further includes:

[0133] The first division generation module is used to divide and generate defect processing areas, and detect defect areas in real time in combination with the convex hull algorithm; or to divide and detect defect locations;

[0134] The second division generation module is used to divide and generate the inner and outer defect detection areas of the weld, and perform image transformation processing on the depth image corresponding to the defect detection area; that is, perform image processing on the depth image corresponding to the defect detection area;

[0135] The third division generation module is used to divide and generate the R-corner defect detection area, and combine the profile contour line method to generate characteristic values ​​corresponding to the weld defects, where the characteristic values ​​include: protrusions, pits, burst points, pinhole depth and length; specifically, weld height, weld width, weld lowest point and length.

[0136] And / or, the second partition generation module further includes:

[0137] A generation module is used to generate weld height position data, and generate weld inner position data and weld outer position data respectively according to the weld height position data; (i.e., respectively search and obtain weld inner position data and weld outer position data);

[0138] The data processing module is used to obtain and process the weld side wall point cloud data and the cap vertex point cloud data respectively.

[0139] In the system solution embodiment of the present invention, the specific details of the method steps involved in the top cover welding defect quality inspection based on multiple cameras have been explained above, that is, the functional modules in the system are used to implement the steps or sub-steps in the above method embodiment, which will not be repeated here.

[0140] To achieve the above objectives, the present invention also provides a multi-camera-based roof welding defect quality inspection platform, such as Figure 4 As shown, it includes a processor, a memory and a control program of a top cover welding defect quality detection platform based on multiple cameras;

[0141] The processor executes the multi-camera-based roof welding defect quality detection platform control program, the multi-camera-based roof welding defect quality detection platform control program is stored in the memory, and the multi-camera-based roof welding defect quality detection platform control program implements the multi-camera-based roof welding defect quality detection method steps. For example:

[0142] S1, obtaining control data for starting camera shooting, and generating image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera;

[0143] S2. Analyze and process the image data corresponding to the top cover welding defect and at different angles, and generate quality inspection data corresponding to the top cover welding defect.

[0144] The specific details of the steps have been explained above and will not be repeated here.

[0145] In the embodiment of the present invention, the built-in processor of the multi-camera-based top cover welding defect quality inspection platform can be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor uses various interfaces and lines to connect various components, and executes or executes programs or units stored in the memory, and calls data stored in the memory to perform various functions and process data of the multi-camera-based top cover welding defect quality inspection;

[0146] The memory is used to store program codes and various data. It is installed in the top cover welding defect quality inspection platform based on multiple cameras, and realizes high-speed and automatic access to programs or data during operation.

[0147] The memory includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electronically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0148] To achieve the above object, the present invention also provides a computer readable storage medium, such as Figure 5 As shown, the computer-readable storage medium stores a multi-camera-based roof welding defect quality detection platform control program, and the multi-camera-based roof welding defect quality detection platform control program implements the multi-camera-based roof welding defect quality detection method steps, for example:

[0149] S1, obtaining control data for starting camera shooting, and generating image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera;

[0150] S2. Analyze and process the image data corresponding to the top cover welding defect and at different angles, and generate quality inspection data corresponding to the top cover welding defect.

[0151] The specific details of the steps have been explained above and will not be repeated here.

[0152] In the description of the embodiments of the present invention, it should be noted that any process or method description in the flowchart or otherwise described herein may be understood as representing a module, fragment or portion of a code comprising one or more executable instructions for implementing steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations, in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0153] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).

[0154] In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0155] In an embodiment of the present invention, in order to achieve the above-mentioned purpose, the present invention further provides a chip system, wherein the chip system includes at least one processor, and when the program instructions are executed in the at least one processor, the chip system executes the steps of the multi-camera-based top cover welding defect quality detection method, for example:

[0156] S1, obtaining control data for starting camera shooting, and generating image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera;

[0157] S2. Analyze and process the image data corresponding to the top cover welding defect and at different angles, and generate quality inspection data corresponding to the top cover welding defect.

[0158] The specific details of the steps have been explained above and will not be repeated here.

[0159] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0160] The present invention obtains control data for starting camera shooting through a method, and generates image data corresponding to the top cover welding defect and at different angles in real time according to the control data; wherein the camera includes a 2D camera and a 3D camera; analyzes and processes the image data corresponding to the top cover welding defect and at different angles, and generates quality inspection data corresponding to the top cover welding defect, as well as a system, platform and storage medium corresponding to the method; combines the principle of image imaging, detects the quality of the weld by a high-quality imaging method and a method combining 3D machine vision and deep learning, and obtains a clear actual image of the defect by improving the imaging quality, and combines the advantages of 2D machine vision, 3D machine vision and deep learning methods to greatly reduce overkill and prevent missed kills. That is, small pinholes, blackening and cracks can be successfully detected, and the risk of missed detection of such defects can be successfully solved.

[0161] In other words, the outlier removal algorithm can successfully remove outlier information, prevent it from affecting subsequent processing, and improve the reliability of detection. The camera installation inclination angle and the movement direction inclination angle can effectively reduce interference signals and improve the accuracy and reliability of detection. The image correction function can effectively reduce overkill, and the algorithm can easily locate the weld, improving the accuracy of detection. The convex hull small defect detection function can easily and accurately detect small defects on the weld. Combined with the weld inside and outside defect algorithms, the defects on the inside and outside of the weld and the side wall can be accurately detected to prevent the detection system from missing. And through the R angle classification processing algorithm, the R angle can be well prevented from missing.

[0162] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting defects in top cover welding based on multiple cameras, characterized in that: The method comprises the following steps: Acquire control data for starting camera shooting, and generate image data corresponding to the top cover welding defect and at different angles in real time according to the control data; Analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and generating quality inspection data corresponding to the top cover welding defect; The analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and generating quality inspection data corresponding to the top cover welding defect, also includes: Acquire point cloud data of image data corresponding to the top cover welding defect, and convert the point cloud data into depth image data; Acquire edge line segment data of the weld, and rotate and correct the defect image in real time according to the edge line segment data; Generate weld area position data, and divide the weld area position data into area types in real time; The generating of the weld area position data and real-time classification of the weld area position data into area types also includes: Divide and generate defect processing areas, and use convex hull algorithm to detect defect areas in real time; Divide and generate the inner and outer defect detection areas of the weld, and perform image transformation processing on the depth image corresponding to the defect detection area; Divide and generate R corner defect detection area, and combine the profile contour method to generate characteristic values ​​corresponding to weld defects, where the characteristic values ​​include: bulge, pit, explosion point, pinhole depth and length; The convex hull algorithm is used to detect small defects. The specific algorithm flow is as follows: Set a small defect detection area ROI in the weld, select a reference area in the ROI, and read the average height value of the reference area; set the upper and lower limits of the extraction of the small defect detection area ROI, and extract the reference area according to the upper and lower limits; use the Otsu method to perform threshold extraction on the reference area to obtain a binary image; perform morphological processing on the binary image to remove small particles and merge areas; extract contours, and perform convex hull processing on those that meet the contour conditions; calculate the convex hull value and determine whether there are defects; take the small defect detection results in each small area as the small defect detection results of the entire weld; The implementation steps of the small defect detection algorithm on the inside and outside of the weld are as follows: Perform image transformation on the depth image, taking the highest point of the weld as the center, divide the weld into two parts, the inner side and the outer side, and then process the side wall and cap point clouds separately; rotate the cap point cloud and the side wall point cloud to the horizontal plane, and then perform depth extraction to remove redundant point cloud information; convert the 16-bit depth image to an 8-bit grayscale image; perform threshold processing and morphological processing; extract contours, extract key feature values, and then determine whether there are defects; R corner defect detection is achieved through the following steps: Quickly locate the R angle position and classify the R angle position into welding start R angle and welding end R angle; use the profile contour line method to calculate the key characteristic values ​​of the weld, such as protrusions, pits, burst points, and the depth and length of pinholes; respectively detect the two types of R angles and compare them with the standard values. Those that meet the conditions are judged as defects, otherwise they are OK values; generate the test results.

2. The method for detecting defects in top cover welding based on multiple cameras according to claim 1 is characterized in that: The step of acquiring control data for starting the camera to shoot, and generating image data corresponding to the top cover welding defect and at different angles in real time according to the control data, further comprises: Image data corresponding to the top cover welding defect is acquired, and a detection mathematical model corresponding to the top cover welding defect is constructed according to the image data.

3. The method for detecting defects in top cover welding based on multiple cameras according to claim 1 is characterized in that: The step of acquiring point cloud data of image data corresponding to the top cover welding defect and converting the point cloud data into depth image data further includes: According to the point cloud data of the image data, outliers in the point cloud data are removed.

4. The method for detecting defects in top cover welding based on multiple cameras according to claim 1 is characterized in that: The division generates inner and outer defect detection areas of the weld, and performs image transformation processing on the depth image corresponding to the defect detection area, and further includes: Generate weld height position data, and generate weld inner position data and weld outer position data respectively according to the weld height position data; The weld side wall point cloud data and cap vertex point cloud data are acquired and processed respectively.

5. A multi-camera-based roof welding defect quality inspection system, characterized in that: The system comprises: A data acquisition and generation unit, used to acquire control data for starting camera shooting, and to generate image data corresponding to the top cover welding defect and at different angles in real time according to the control data; A data analysis and processing generating unit, used for analyzing and processing the image data corresponding to the top cover welding defect and at different angles, and generating quality inspection data corresponding to the top cover welding defect; The data analysis processing generation unit further includes: An acquisition conversion module is used to acquire point cloud data of image data corresponding to the top cover welding defect, and convert the point cloud data into depth image data; An acquisition rotation correction module is used to acquire edge line segment data of the weld and to rotate and correct the defect image in real time according to the edge line segment data; A generation and division module is used to generate weld area position data and divide the weld area position data into area types in real time; And / or, generating a partitioning module further includes: The first division generation module is used to divide and generate defect processing areas, and detect defect areas in real time in combination with a convex hull algorithm; The second division generation module is used to divide and generate the inner and outer defect detection areas of the weld, and perform image transformation processing on the depth image corresponding to the defect detection area; The third division generation module is used to divide and generate the R corner defect detection area, and generate the characteristic values ​​corresponding to the weld defects in combination with the profile contour method, wherein the characteristic values ​​include: bulges, pits, burst points, pinhole depth and length; The convex hull algorithm is used to detect small defects. The specific algorithm flow is as follows: Set a small defect detection area ROI in the weld, select a reference area in the ROI, and read the average height value of the reference area; set the upper and lower limits of the extraction of the small defect detection area ROI, and extract the reference area according to the upper and lower limits; use the Otsu method to perform threshold extraction on the reference area to obtain a binary image; perform morphological processing on the binary image to remove small particles and merge areas; extract contours, and perform convex hull processing on those that meet the contour conditions; calculate the convex hull value and determine whether there are defects; take the small defect detection results in each small area as the small defect detection results of the entire weld; The implementation steps of the small defect detection algorithm on the inside and outside of the weld are as follows: Perform image transformation on the depth image, taking the highest point of the weld as the center, divide the weld into two parts, the inner side and the outer side, and then process the side wall and cap point clouds separately; rotate the cap point cloud and the side wall point cloud to the horizontal plane, and then perform depth extraction to remove redundant point cloud information; convert the 16-bit depth image to an 8-bit grayscale image; perform threshold processing and morphological processing; extract contours, extract key feature values, and then determine whether there are defects; R corner defect detection is achieved through the following steps: Quickly locate the R angle position and classify the R angle position into welding start R angle and welding end R angle; use the profile contour line method to calculate the key characteristic values ​​of the weld, such as protrusions, pits, burst points, and the depth and length of pinholes; respectively detect the two types of R angles and compare them with the standard values. Those that meet the conditions are judged as defects, otherwise they are OK values; generate the test results.

6. The multi-camera-based roof welding defect quality inspection system according to claim 5 is characterized in that: The data acquisition generation unit further includes: A model building module, used to obtain image data corresponding to the top cover welding defect, and build a detection mathematical model corresponding to the top cover welding defect according to the image data; And / or, obtaining a conversion module further includes: A data removal module, used for removing outliers in the point cloud data according to the point cloud data of the image data; And / or, the second partition generation module further includes: A generating module, used to generate weld height position data, and generate weld inner position data and weld outer position data according to the weld height position data; The data processing module is used to obtain and process the weld side wall point cloud data and the cap vertex point cloud data respectively.

7. A multi-camera-based roof welding defect quality inspection platform, characterized in that: It includes a processor, a memory and a control program of a top cover welding defect quality inspection platform based on multiple cameras; Wherein the processor executes the multi-camera based top cover weld defect quality detection platform control program, the multi-camera based top cover weld defect quality detection platform control program is stored in the memory, and the multi-camera based top cover weld defect quality detection platform control program implements the multi-camera based top cover weld defect quality detection method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a multi-camera-based top cover weld defect quality detection platform control program, and the multi-camera-based top cover weld defect quality detection platform control program implements the multi-camera-based top cover weld defect quality detection method as described in any one of claims 1 to 4.

Images