A method and system for detecting the weld appearance of energy storage box based on machine vision

Abstract

The present invention relates to the technical field of energy storage box weld detection, and specifically discloses a method and system for energy storage box weld appearance detection based on machine vision. The invention obtains a weld appearance image and extracts key features therefrom, identifies defects based on the key features, and determines whether there are defects on the weld surface. Then, the internal image data of the weld is obtained by using non-destructive testing technology, and three-dimensional reconstruction is performed after preprocessing to generate a three-dimensional model. Internal defects are identified through image processing. Finally, the weld appearance image is matched with feature points in the internal structure data, and adjustments are made to ensure that the error is within an acceptable range, and the data is fused to generate a complete view of the weld. The technical solution of the present invention realizes comprehensive detection and comprehensive evaluation of weld surface features and internal structure information, and significantly improves the accuracy and reliability of weld detection.

Description

Technical Field

[0001] The present invention relates to the technical field of energy storage box weld detection, and in particular to a method and system for detecting the weld appearance of an energy storage box based on machine vision. Background Art

[0002] With the development of energy storage technology, the welding quality of energy storage boxes has become an important factor affecting the safety of energy storage systems. At present, weld detection mainly relies on manual visual inspection and some traditional non-destructive testing technologies, such as ultrasonic testing and magnetic particle testing. However, these methods often have problems such as low efficiency and strong subjectivity. In recent years, image processing technology has been gradually applied to weld detection due to its high efficiency and objectivity, but it still faces problems such as separation of weld surface and internal defect detection, incomplete defect feature extraction, and poor consistency of detection results.

[0003] Existing weld inspection methods mainly include manual visual inspection, which relies on the experience and vision of the inspectors and is easily affected by subjective factors; ultrasonic inspection can detect internal defects of welds, but is not effective for surface defect detection; magnetic particle inspection is suitable for detecting surface and near-surface defects, but cannot detect internal defects; image processing technology uses computer vision technology to detect weld surface defects, but lacks effective combination with internal defect detection.

[0004] These methods each have certain limitations. For example, manual visual inspection is inefficient and prone to misjudgment. Although traditional non-destructive testing technologies such as ultrasonic and magnetic particle testing can detect internal or surface defects, they cannot simultaneously achieve comprehensive detection of both weld surface and internal defects. Image processing technology has made certain progress in weld surface defect detection, but there is still room for improvement in the comprehensiveness of defect feature extraction and the consistency of test results.

[0005] The present invention aims to overcome the shortcomings of the prior art and proposes a detection method for comprehensively processing weld surface and internal defects. This method can not only effectively extract the surface features of the weld, but also perform a three-dimensional scan of the inside of the weld and fuse the information of the two to ultimately generate a complete view of the weld. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for detecting the appearance of a weld of an energy storage box based on machine vision. The technical solution of the present invention relates to a method for integrated detection of weld surface and internal defects, which specifically includes the following steps: firstly, obtaining an appearance image of the weld by an image acquisition device, and then extracting the features of the weld such as the contour, width and color by an image processing algorithm; performing defect identification based on these features, such as cracks and pits; finally, calculating a weld feature vector and comparing it with a standard normal weld feature vector to determine whether there are defects on the weld surface, and obtaining the internal image data of the weld by using non-destructive testing technology (such as ultrasonic testing, X-ray testing, etc.), and pre-processing the weld; Then, three-dimensional reconstruction is performed to generate a three-dimensional model of the weld; defects inside the weld are identified by performing image processing on the three-dimensional model, and the weld appearance image obtained above is matched with characteristic points in the weld internal structure data obtained by stereo scanning technology; matching errors are calculated and necessary adjustments are made to ensure that the errors are within a predetermined acceptable range; the matched data are fused to generate a complete view containing weld surface features and internal structure information. The technical solution of the present invention overcomes the problem of separation of weld surface and internal defect detection in the prior art, and realizes integrated detection and comprehensive evaluation of weld surface features and internal structure information, thereby significantly improving the accuracy and reliability of weld detection.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A method and system for detecting the appearance of a weld of an energy storage box based on machine vision, comprising the following steps:

[0009] Process the weld appearance image of the energy storage box to obtain the key features of the weld;

[0010] Among them, the key characteristics of the weld include the weld profile, width and color;

[0011] Process the key features of the weld to obtain the surface defects of the weld;

[0012] Acquire the three-dimensional scanning data of the weld and obtain the three-dimensional model of the weld;

[0013] Based on the three-dimensional model of the weld, the defect area inside the weld is obtained;

[0014] The planar image and stereoscopic scanning data of the weld are fused to generate a complete view of the weld.

[0015] As a further solution of the present invention: after the energy storage box completes the welding task, an image of the weld appearance of the energy storage box is obtained;

[0016] Based on the weld appearance image of the energy storage box, position information of each pixel in the weld appearance image is obtained;

[0017] Each pixel in the weld appearance image is mapped as a point into the plane coordinate system to obtain the coordinate position of each pixel, which is recorded as a pixel point;

[0018] The weld appearance image and all pixel points in the weld appearance image are processed to obtain a complete outline of the weld.

[0019] As a further solution of the present invention: based on the complete contour of the weld, all pixel points (x i ,y i ) to obtain the minimum x value, maximum x value, minimum y value and maximum y value;

[0020] Where i is the number of pixels on the weld contour, i=1,2,...,n;

[0021] The maximum x value and the minimum x value are calculated as the difference to obtain the width of the weld.

[0022] As a further solution of the present invention: the process of obtaining the surface defects of the weld is:

[0023] Obtaining a number of actual image samples representing a particular type of defect;

[0024] Among them, specific types of defects include but are not limited to cracks, pits, etc.;

[0025] The defective weld area in the actual image sample is cropped to obtain a template image;

[0026] Process all template images to make the sizes of all template images consistent;

[0027] Integrate all processed template images to obtain a defect template library;

[0028] Based on the weld area in the energy storage box weld appearance image, the weld area is separated from the background in the energy storage box weld appearance image to obtain a separate weld area image;

[0029] Compare the weld area image with one or more template images in the defect template library to obtain a correlation coefficient between the weld area image and the template image;

[0030] comparing the correlation coefficient to a correlation coefficient threshold;

[0031] If the correlation coefficient is greater than the correlation coefficient threshold, there is a defect in the template image at that position;

[0032] For each location where the correlation coefficient is greater than the correlation coefficient threshold, the location is marked as a defective location.

[0033] As a further solution of the present invention: the calculation process of the correlation coefficient between the weld area image and the template image is:

[0034] Obtain the pixel value of each pixel in the weld area image;

[0035] According to the mean calculation formula, the pixel value of each pixel in the weld area image is processed to obtain the average pixel value J of the weld area image. I ;

[0036] Get the pixel value of each pixel in the template image;

[0037] According to the mean calculation formula, the pixel value of each pixel in the template image is processed to obtain the average pixel value J of the template image. T ;

[0038] By formula The correlation coefficient XGX is calculated;

[0039] Among them, T is the template image, I is the weld area image, and I p (x, y) is the pixel value of the weld area image at position p, T(x, y) is the pixel value of the template image, ∑ x,y is the sum of all pixels of the template image, XGX is the sub-image I at a certain position p in the template image T and the weld area image I p The correlation coefficient between .

[0040] As a further solution of the present invention: the second method for obtaining the surface defects of the weld is:

[0041] Based on the weld width and weld color features in the weld appearance image of the energy storage box;

[0042] Among them, the color feature of the weld is the color histogram;

[0043] The weld width and color histogram are combined to obtain a set of weld feature vectors;

[0044] Obtain a set of feature vectors of normal welds;

[0045] The characteristic vector of the weld is calculated to be different from the characteristic vector of a normal weld to obtain a characteristic vector deviation value;

[0046] The squares of all characteristic vector deviation values ​​are summed up and the obtained value is squared to obtain the error value between the characteristic vector of the weld and the characteristic vector of the normal weld;

[0047] comparing the error value to an error value threshold;

[0048] If the error value is greater than the error value threshold, it means that there is a defect in the weld of the energy storage box.

[0049] As a further solution of the present invention: the process of obtaining the defective area inside the weld is:

[0050] After the energy storage box completes the welding task, the image data inside the weld is obtained and recorded as three-dimensional scanning data;

[0051] The three-dimensional scanning data is reconstructed in three dimensions to obtain a three-dimensional model of the weld;

[0052] The three-dimensional model of the weld is converted into grayscale to obtain the grayscale value of each pixel in the three-dimensional model;

[0053] All gray values ​​in the three-dimensional model of the weld are processed to obtain a histogram of the gray image;

[0054] Perform threshold analysis on the histogram of the grayscale image to obtain the optimal threshold for image segmentation;

[0055] Based on the optimal threshold, the three-dimensional model is segmented according to the threshold to obtain areas above the threshold and areas below the threshold;

[0056] The area above the threshold is marked as a defective area and recorded as the defective area of ​​the weld.

[0057] As a further solution of the present invention, the process of fusing the planar image and the stereoscopic scanning data of the weld is as follows:

[0058] Based on the weld appearance image of the energy storage box, the starting point, the ending point and the corner point of the weld in the weld appearance image are obtained;

[0059] The starting point, ending point and corner point of the weld are used as reference points and marked in the weld appearance image;

[0060] Based on the reference points marked in the weld appearance image, the starting point, the end point and the corner point of the weld are marked as reference points in the stereo scan data;

[0061] Obtain pixel coordinates of reference points in the weld appearance image;

[0062] Obtain pixel coordinates of reference points in stereo scanning data;

[0063] Based on the pixel coordinates of the reference point in the stereo scanning data, the starting point of the weld in the stereo scanning data is taken as the origin, the weld extension direction is the X axis, the direction perpendicular to the weld is the Y axis, and the direction perpendicular to the weld plane is the Z axis, to construct a three-dimensional coordinate system for the stereo scanning data;

[0064] Based on the three-dimensional coordinate system of the stereoscopic scanning data and the pixel coordinates of the reference points in the weld appearance image, a coordinate conversion algorithm is used to convert the coordinates in the weld appearance image into the same three-dimensional coordinate system as the stereoscopic scanning data, which is recorded as the three-dimensional coordinate system of the appearance image;

[0065] Processing the three-dimensional coordinate system of the appearance image to obtain the coordinate points of the weld contour and the coordinate points of the weld defects in the three-dimensional coordinate system of the appearance image, which are recorded as the appearance image feature points;

[0066] Processing the three-dimensional coordinate system of the stereoscopic scanning data to obtain coordinate points of the weld contour and the weld defect in the three-dimensional coordinate system of the stereoscopic scanning data, which are recorded as stereoscopic scanning feature points;

[0067] All appearance image feature points are matched with corresponding stereo scanning feature points to obtain a three-dimensional coordinate system after the weld appearance image and the stereo scanning data are fused.

[0068] As a further solution of the present invention: based on the three-dimensional coordinate system after the weld appearance image and the stereoscopic scanning data are fused, all the appearance image feature points and the corresponding stereoscopic scanning feature points are processed according to the Euclidean distance calculation formula to obtain the Euclidean distance between each appearance image feature point and the corresponding stereoscopic scanning feature point;

[0069] Based on, through the formula The average distance error MDE between the appearance image feature points and the corresponding stereo scan feature points is calculated;

[0070] Among them, D u is the distance between each pair of feature points, u is the number of feature points, u=1,2,…,N;

[0071] Compare the mean distance error MDE with the mean distance error threshold;

[0072] If the average distance error MDE is greater than the average distance error threshold, re-matching is performed;

[0073] If the average distance error MDE is less than or equal to the average distance error threshold, a matching qualified signal is generated;

[0074] Based on the matching qualified signals, the three-dimensional coordinate system of the stereoscopic scanning data is fused with the three-dimensional coordinate system of the appearance image to generate a complete view of the weld;

[0075] Among them, the complete view of the weld includes the weld profile, surface defects, weld shape and weld view of internal defects.

[0076] As a further solution of the present invention: a machine vision-based energy storage box weld appearance detection system, characterized in that it includes a plane image acquisition detection module, a three-dimensional data scanning module, a stereo scanning data analysis module, a plane and three-dimensional fusion module and a visualization module;

[0077] The plane image acquisition and detection module is used to process the weld appearance image of the energy storage box to obtain the key features of the weld;

[0078] Among them, the key characteristics of the weld include the weld profile, width and color;

[0079] Process the key features of the weld to obtain the surface defects of the weld;

[0080] The three-dimensional data scanning module is used to obtain the three-dimensional scanning data of the weld and obtain the three-dimensional model of the weld;

[0081] The stereo scanning data analysis module obtains the defect area inside the weld based on the three-dimensional model of the weld;

[0082] The plane and three-dimensional fusion module is used to fuse the plane image and three-dimensional scanning data of the weld to generate a complete view of the weld;

[0083] The visualization module is used to display the complete 3D model of the weld.

[0084] Beneficial effects of the present invention:

[0085] (1) The present invention provides an integrated detection method for weld surface and internal defects, which has the beneficial effect of significantly improving detection efficiency and accuracy. Traditional detection methods often require separate surface and internal defect detection, which is not only time-consuming but also prone to misjudgment due to improper operation. The present invention adopts an integrated detection process to simultaneously obtain weld surface features and internal structure information, effectively avoiding the error accumulation problem that may be caused by step-by-step detection. Through advanced image processing technology and three-dimensional reconstruction algorithms, it can quickly and accurately identify tiny defects on the weld. In addition, the present invention also introduces a machine learning algorithm to train historical detection data, further optimizing the detection model to enable it to have higher recognition accuracy and robustness. This highly integrated detection method greatly shortens the detection cycle, reduces labor costs, and effectively reduces the occurrence of missed detections and false detections;

[0086] (2) Another major advantage of the present invention is that it realizes the automation and intelligence of the weld inspection process. Traditional manual inspection relies on the experience and technical level of the inspectors, and is greatly affected by subjective factors, making it difficult to ensure the consistency of each inspection. The present invention collects weld image data through automated equipment and uses advanced image recognition algorithms to automatically complete feature extraction and defect classification. The entire process does not require human intervention, greatly reducing the impact of human factors. At the same time, with the help of deep learning technology, the system can continuously learn new defect types and patterns and continuously optimize its own performance. Such automated and intelligent design not only improves the inspection quality, but also makes the inspection process more stable and reliable;

[0087] (3) The present invention also significantly improves the safety and reliability of welded structures. By conducting comprehensive and detailed inspections of welds, potential defects and hidden dangers can be discovered in a timely manner, so that corresponding repair measures can be taken. The integrated inspection method can effectively avoid the risks caused by missing important information by a single inspection method. For example, in the fields of aerospace, bridge construction, etc., welding parts are often key load-bearing points, and their quality is directly related to the safety of the overall structure. The inspection scheme provided by the present invention can ensure that the quality of welds meets high standards and helps prevent accidents. In addition, due to the use of advanced image processing and data analysis technologies, the inspection results are more accurate and reliable, providing a reliable basis for subsequent repairs and maintenance. Therefore, the present invention is of great significance for improving the overall performance of welded structures and has made positive contributions to ensuring public safety and promoting industrial development. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] The present invention will be further described below in conjunction with the accompanying drawings.

[0089] Figure 1 It is a flow chart of specific steps of a method and system for detecting the appearance of a weld of an energy storage box based on machine vision of the present invention;

[0090] Figure 2 It is a flow chart of obtaining the defective area inside the weld in the present invention;

[0091] Figure 3 It is a flow chart of fusing the planar image and stereoscopic scanning data of the weld in the present invention. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example 1

[0093] See also Figure 1 As shown, the present invention is a method for detecting the appearance of a weld of an energy storage box based on machine vision, comprising the following steps:

[0094] Process the weld appearance image of the energy storage box to obtain the key features of the weld;

[0095] Among them, the key characteristics of the weld include the weld profile, width and color;

[0096] Process the key features of the weld to obtain the surface defects of the weld;

[0097] Acquire the three-dimensional scanning data of the weld and obtain the three-dimensional model of the weld;

[0098] Based on the three-dimensional model of the weld, the defect area inside the weld is obtained;

[0099] The planar image and stereoscopic scanning data of the weld are fused to generate a complete view of the weld. Example 2

[0100] In a specific embodiment:

[0101] After the energy storage box completes the welding task, obtain the weld appearance image of the energy storage box;

[0102] Based on the weld appearance image of the energy storage box, position information of each pixel in the weld appearance image is obtained;

[0103] Map each pixel in the weld appearance image as a point into the plane coordinate system (x, y) to obtain the coordinate position of each pixel, which is recorded as a pixel point;

[0104] Process the weld appearance image and all pixel points in the weld appearance image to obtain the complete outline of the weld;

[0105] It should be noted that the weld appearance image can be processed by the edge detection algorithm to identify the boundary of the weld. Once the boundary is found, the complete outline of the weld can be depicted by the contour tracking technology. The complete outline of the weld can help us determine the specific shape and position of the weld.

[0106] Based on the complete outline of the weld, the coordinates of all pixel points on the weld outline are obtained (x i ,y i );

[0107] Where i is the number of pixels on the weld contour, i=1,2,...,n;

[0108] All pixel points on the weld contour are processed to obtain the minimum x value, maximum x value, minimum y value and maximum y value;

[0109] Among them, the minimum x value x min =min(x i ), the maximum x value x max =max(x i ), minimum y value y min =min(y i ), maximum y value y max =max(y i );

[0110] Calculate the difference between the maximum x value and the minimum x value to obtain the width of the weld;

[0111] It should be noted that the above width calculation is performed by calculating the bounding box of the weld contour and using its width as the weld width;

[0112] Based on all the pixel points on the weld contour (x i ,y i ), through the formula Calculate the pixel points on the weld contour (x i ,y i ) and pixel (x j ,y j )The distance JL(i,j) between them;

[0113] Where j is the number of pixels on the weld contour, j=1,2,...,n;

[0114] For all points on the weld contour (x i ,y i ), find the point with the longest distance (x i ,y i ), get JL max =max(JL(i,j));

[0115] The width of the weld is min(JL max );

[0116] It should be noted that, in addition to calculating the bounding box of the weld contour and using its width as the weld width, the distance from each pixel point on the weld contour to its farthest pixel point can also be calculated, and the minimum value is taken as an estimate of the weld width;

[0117] Based on the weld contour and weld width, the weld appearance image is processed to obtain the complete area of ​​the weld, which is recorded as the weld area;

[0118] Process the color of the weld area and extract the average color and color histogram of the weld area;

[0119] It should be noted that by processing the image color through machine vision technology, the average color value and color histogram of the weld area can be calculated. Calculating the average color value of the weld area will help to subsequently identify changes or contamination of the weld material. By comparing the color difference between the weld area and the surrounding area, potential defects can be detected, such as foreign body contamination or inconsistent weld color. By constructing a color histogram of the weld area, the distribution of different color intensities can be quantified, which will help to subsequently identify abnormal color changes in the weld area. Example 3

[0120] Based on the weld contour, weld width and weld color histogram extracted from the energy storage box weld appearance image, the energy storage box weld is identified for defects, specifically:

[0121] In a specific embodiment:

[0122] Obtaining a number of actual image samples representing a particular type of defect;

[0123] Among them, specific types of defects include but are not limited to cracks, pits, etc.;

[0124] The defective weld area in the actual image sample is cropped to obtain a template image;

[0125] Preprocess all template images to make the sizes of all template images consistent;

[0126] Integrate all preprocessed template images to obtain a defect template library;

[0127] Based on the weld area in the energy storage box weld appearance image, the weld area is separated from the background in the energy storage box weld appearance image to obtain a separate weld area image;

[0128] Compare the weld area image with one or more template images in the defect template library, specifically:

[0129] Obtain the pixel value of each pixel in the weld area image;

[0130] According to the mean calculation formula, the pixel value of each pixel in the weld area image is processed to obtain the average pixel value J of the weld area image. I ;

[0131] Get the pixel value of each pixel in the template image;

[0132] According to the mean calculation formula, the pixel value of each pixel in the template image is processed to obtain the average pixel value J of the template image. T ;

[0133] By formula The correlation coefficient XGX is calculated;

[0134] Among them, T is the template image, I is the weld area image, and I p (x, y) is the pixel value of the weld area image at position p, T(x, y) is the pixel value of the template image, ∑ x,y is the sum of all pixels of the template image, XGX is the sub-image I at a certain position p in the template image T and the weld area image I p The correlation coefficient between

[0135] It should be noted that the above method extracts the sub-image I at each possible position p in the weld area image. p , by calculating I p The correlation coefficient value between the image and the template image T is used to analyze whether there is a high degree of similarity between the two images, so as to determine whether there is a defect in the weld area image;

[0136] In the above method, the size of the template image is much smaller than the target image. This is because the idea of ​​analyzing through the template image is to find the position of the template image in the larger target image. Specifically, the template image is usually a small picture or an image of a specific pattern, and its size is much smaller than the target image (weld area image), while the target image (weld area image) is a larger image that may contain the template image.

[0137] By calculating the similarity between the sub-image at each possible position of the weld area image and the template image, if the template image is very similar to a certain part of the weld area image, the similarity score calculated at that part will be very high;

[0138] In the above calculation, the template image T and the sub-image I at a certain position p in the weld area image are p As a matrix consisting of pixel values, where T(x,y) and I p (x,y) distribution represents the template image T and the sub-image I p The pixel value at the coordinate point (x, y) is the sub-image of the weld area image with the same size as the template image when sliding the template image T in the weld area image I, which is I p , since sub-image I p The size of is the same as that of the template image T, so that pixel-level comparison can be performed directly;

[0139] Based on the template image T and the sub-image I at position p in the weld area image I p The correlation coefficient between them is XGX;

[0140] If the correlation coefficient XGX is compared with the correlation coefficient threshold;

[0141] If the correlation coefficient XGX is greater than the correlation coefficient threshold, it means that a defect in the template image is found at that position;

[0142] For each position p where the correlation coefficient XGX is greater than the correlation coefficient threshold, the p position is marked as a defect position;

[0143] Based on the weld width and weld color histogram in the weld appearance image of the energy storage box;

[0144] The weld width and color histogram are combined to obtain a set of weld feature vectors Fc=[W,H1,H2,...,H m ];

[0145] Where W is the weld width, H is the color histogram, and m is the number of channels of the color histogram, m=1,2,...,m;

[0146] Obtain a set of characteristic vectors Fz of normal welds;

[0147] By formula Calculate the error value OSJ between the characteristic vector Fc of the weld and the characteristic vector Fz of the normal weld;

[0148] Among them, m+1 is the dimension of the feature vector, that is, the width plus the number of channels of the color histogram, and k is the number of elements in the feature vector, k=1,2,...,m+1;

[0149] Compare the error value OSJ between the characteristic vector Fc of the weld and the characteristic vector Fz of the normal weld with the error value threshold;

[0150] If the error value OSJ is greater than the error value threshold, it means that there is a defect in the weld of the energy storage box;

[0151] It should be noted that by calculating I pThe method of analyzing whether there is a high similarity between the two images and whether there is a defect in the weld area image by using the correlation coefficient value between the weld and the template image T is used to identify specific defect patterns, such as cracks, pits, etc. This is very effective for identifying common defect types. By analyzing the characteristic vector Fc of the weld and the characteristic vector Fz of the normal weld, subtle changes that do not have obvious patterns but still affect the quality of the weld can be identified, such as changes in color, texture, and shape. Calculating the correlation coefficient value alone may be affected by the image quality, such as fiber changes, slight changes in the weld surface, etc., which may lead to matching failures. By analyzing the characteristic vector, the characteristics of the weld can be analyzed from multiple dimensions, which can provide useful information even in the case of poor image quality.

[0152] Real-time Example 4

[0153] See also Figure 2 As shown, in a specific embodiment:

[0154] After the energy storage box is welded, non-destructive testing technology (including but not limited to X-ray, CT scanning, etc.) is used to obtain image data inside the weld, which is recorded as three-dimensional scanning data;

[0155] Preprocessing the collected stereo scanning data;

[0156] Wherein, preprocessing includes but is not limited to removing noise and enhancing contrast;

[0157] The pre-processed stereo scanning data is three-dimensionally reconstructed to obtain a three-dimensional model of the weld;

[0158] Among them, the form of 3D model includes point cloud form or surface network form

[0159] The three-dimensional model of the weld is converted into grayscale. For each pixel point P(x, y, z), the grayscale value G is calculated as G=R×0.299+G×0.587+B×0.114, where R, G, and B represent the values ​​of the red, green, and blue channels of the pixel point, respectively.

[0160] All gray values ​​in the three-dimensional model of the weld are processed to obtain a histogram of the gray image;

[0161] The histogram is analyzed using Otsu's method to obtain the optimal threshold for image segmentation;

[0162] Among them, Otsu's method is to determine the optimal threshold by maximizing the inter-class variance between the foreground and the background. The foreground is the area in the image that needs to be identified or separated. For example, in a photo containing an object, the object itself is regarded as the foreground. The foreground usually has a high contrast or a specific grayscale pattern. The background is the other parts of the image except the foreground. These are usually areas that do not require special attention. For example, in a photo containing an object, the environment around the object or the featureless area is regarded as the background. The background is usually more uniform than the foreground and has a lower contrast.

[0163] The calculation formula of Otsu's method is: ,T is the optimal threshold, μ0(T) and μ1(T) are the average grayscale values ​​of the foreground and background respectively, which are obtained by processing all the grayscale values ​​of the foreground and background regions respectively using the mean calculation formula, ω0(T) and ω1(T) are the preset weights of the foreground and background respectively, μT is the average grayscale value of the entire image, which is obtained by processing all the grayscale values ​​of the entire image using the mean calculation formula;

[0164] Based on the optimal threshold, the point cloud or mesh in the three-dimensional model is segmented according to the threshold to obtain areas above the threshold and areas below the threshold;

[0165] The area above the threshold is marked as a defective area and recorded as the defective area of ​​the weld;

[0166] In the three-dimensional model, the defect area of ​​the weld is measured to obtain the size data of the defect;

[0167] The defect size data includes length, width and depth. The length is the maximum distance in the defect extension direction, the width is the maximum distance perpendicular to the length direction, and the depth is the distance from the weld surface to the bottom of the defect.

[0168] Based on the defect size data, the contour of the defect area is identified through image processing technology to obtain the contour area and perimeter of the defect area and generate the actual shape of the defect area;

[0169] Differentiate the actual shape of the defect area from the ideal circle or ellipse;

[0170] Among them, the roundness formula for differential analysis of the actual shape of the defect area and the ideal circle is: , MA is the contour area of ​​the defect area, ZP is the perimeter of the defect area, and YD is the roundness value. If the roundness value is greater than the roundness value threshold, it means that the contour of the defect area is close to a circle, and vice versa;

[0171] The difference analysis between the actual shape of the defect area and the ellipse is to calculate the ratio of the width to the length of the outline of the defect area to obtain the ellipse value. If the ellipse value is greater than the ellipse value threshold, it means that the outline of the defect area is close to an ellipse, and vice versa.

[0172] The aspect ratio of the defect area is obtained by calculating the ratio of the length to the width of the contour of the defect area. If the aspect ratio is greater than the aspect ratio threshold, it means that the contour of the defect area deviates from a circle or an ellipse.

[0173] Comprehensively analyze the roundness value, ellipticity value and aspect ratio of the actual shape of the above defect area;

[0174] If the roundness value and ellipticity value are greater than the threshold, and the aspect ratio is less than or equal to the threshold, the actual shape of the defect area is considered regular;

[0175] Otherwise, the actual shape of the defect area is considered irregular;

[0176] It should be noted that by analyzing the regularity or irregularity of the shape of the defect area, the type of weld defect can be obtained. For example, if the actual shape of the defect area is regular, the weld defect is pores or slag inclusions. If the actual shape of the defect area is irregular, the weld defect is cracks or lack of fusion. Example 5

[0177] See also Figure 3 As shown, based on the above data fusion processing of the results obtained through the weld appearance image processing of the energy storage box (including weld contour and surface defects) and the results obtained through the stereo scanning data processing (including weld shape and internal defects), all the inspection data are analyzed and the weld is evaluated as a whole. Specifically:

[0178] Based on the weld appearance image of the energy storage box, a starting point of the weld, an end point of the weld, and a corner point of the weld in the weld appearance image are obtained;

[0179] The starting point, ending point and corner point of the weld are used as reference points and marked in the weld appearance image;

[0180] It should be noted that the reference points in the weld appearance image are three non-collinear points selected in the plane image. These three points are not on the same straight line. The purpose of this is to ensure that the spatial correspondence between the weld appearance image and the stereoscopic scanning data can be accurately established, that is, the correspondence between the plane image and the three-dimensional stereo image. Non-collinear reference points can improve the accuracy of coordinate transformation because they can provide more spatial information and can perform coordinate transformation more accurately. Moreover, the use of three non-collinear points can ensure the uniqueness of coordinate transformation, that is, there is only one way to transform the coordinates in the plane image into the three-dimensional coordinate system, and collinear points may cause uncertainty or multiple solutions during coordinate transformation, while non-collinear points can avoid this situation. The selection of non-collinear points should ensure that these points can be clearly seen in both the plane image and the three-dimensional stereo image, and select points that can represent the overall characteristics of the weld, such as the starting point of the weld, the end point of the weld, and the corner point of the weld;

[0181] Based on the reference points marked in the weld appearance image, the starting point, the end point and the corner point of the weld are marked as reference points in the stereo scan data;

[0182] Obtain pixel coordinates of reference points in the weld appearance image;

[0183] Obtain pixel coordinates of reference points in stereo scanning data;

[0184] The starting point of the weld in the stereoscopic scanning data is taken as the origin, the extension direction of the weld is the X-axis, the direction perpendicular to the weld is the Y-axis, and the direction perpendicular to the weld plane is the Z-axis, to construct a three-dimensional coordinate system for the stereoscopic scanning data;

[0185] Based on the three-dimensional coordinate system of the stereoscopic scanning data, a coordinate conversion algorithm is used to convert the coordinates in the weld appearance image into the same three-dimensional coordinate system as the stereoscopic scanning data, which is recorded as the three-dimensional coordinate system of the appearance image;

[0186] It should be noted that the significance of coordinate transformation is to ensure that the plane image and the stereoscopic scanning data are in the same coordinate system, so that these data are comparable; through coordinate transformation, the accuracy of data fusion can be improved, and the quality of the weld can be evaluated more accurately; at the same time, it can also integrate the features in the plane image (weld contour, surface defects) with the features in the stereoscopic scanning data (weld shape, internal defects) to form a more complete weld view. Its role is mainly reflected in the following aspects: first, it allows the features in the plane image to be combined with the features in the stereoscopic scanning data to form a comprehensive weld quality report; second, in a unified coordinate system, the geometric characteristics and internal defects of the weld can be more accurately evaluated; finally, a detailed inspection report can be generated, which includes the comprehensive analysis results of the two-dimensional and three-dimensional data and suggestions for improvement.

[0187] Processing the three-dimensional coordinate system of the appearance image, marking the coordinate points belonging to the weld contour and the coordinate points belonging to the weld defects in the three-dimensional coordinate system of the appearance image;

[0188] All weld contour coordinate points in the three-dimensional coordinate system of the appearance image are taken as main feature points, and all weld defect coordinate points in the three-dimensional coordinate system of the appearance image are taken as auxiliary feature points;

[0189] Processing the three-dimensional coordinate system of the stereoscopic scanning data, marking the coordinate points belonging to the weld contour and the coordinate points belonging to the weld defects in the three-dimensional coordinate system of the stereoscopic scanning data;

[0190] All weld contour coordinate points in the three-dimensional coordinate system of the stereoscopic scanning data are taken as main feature points, and all weld defect coordinate points in the three-dimensional coordinate system of the stereoscopic scanning data are taken as auxiliary feature points;

[0191] Match all feature points in the three-dimensional coordinate system of the stereo scan data with all feature points in the three-dimensional coordinate system of the appearance image one by one;

[0192] Evaluate the corresponding matching results of all feature points in the three-dimensional coordinate system of the stereo scan data and all feature points in the three-dimensional coordinate system of the appearance image, and analyze whether the matching accuracy error is within an acceptable range. Specifically:

[0193] All feature points in the three-dimensional coordinate system of the appearance image and all feature points in the three-dimensional coordinate system of the corresponding stereo scan data are processed according to the Euclidean distance calculation formula to obtain the Euclidean distance between all feature points in the three-dimensional coordinate system of the appearance image and all feature points in the three-dimensional coordinate system of the corresponding stereo scan data;

[0194] The calculation formula of Euclidean distance is: , where (X1, Y1, Z1) and (X2, Y2, Z2) are the coordinates of the feature points in the three-dimensional coordinate system of the appearance image and the three-dimensional coordinate system of the stereo scan data, respectively, and u is the number of feature points, u=1,2,…,N;

[0195] Based on the distance D of each pair of feature points u , the average distance error between the feature points in the three-dimensional coordinate system of the appearance image and the feature points in the three-dimensional coordinate system of the stereo scan data is calculated by the average distance error formula;

[0196] Among them, the calculation formula of the average distance error formula is: , N is the total number of feature points;

[0197] Compare the mean distance error MDE with the mean distance error threshold;

[0198] If the average distance error MDE is greater than the average distance error threshold, it means that the corresponding matching errors of all feature points in the three-dimensional coordinate system of the stereoscopic scanning data and the three-dimensional coordinate system of the appearance image are too large, and the matching is performed again;

[0199] If the average distance error MDE is less than or equal to the average distance error threshold, it means that the corresponding matching errors of all feature points in the three-dimensional coordinate system of the stereoscopic scanning data and the three-dimensional coordinate system of the appearance image are within an acceptable range, and a matching qualified signal is generated;

[0200] Based on the matching qualified signals, the three-dimensional coordinate system of the stereoscopic scanning data is fused with the three-dimensional coordinate system of the appearance image to synthesize the comprehensive view of the weld contour, surface defects, weld shape and internal defects to form a complete view of the weld, that is, the three-dimensional model view;

[0201] Based on the complete view of the weld, the visualization module displays different perspectives of the complete 3D model of the weld, so as to observe the weld status in all directions;

[0202] It should be noted that by integrating plane images and stereoscopic scanning data, a complete view of the weld can be obtained, including multi-dimensional information such as the weld's geometric features, surface quality, and internal structure. This effect is achieved by using three-dimensional rendering technology in computer graphics to integrate the weld's appearance features and internal structure information into a three-dimensional model to generate a complete view of the weld, showing the weld's contour, surface defects, shape, internal defects and other information. This method improves the accuracy of weld detection by complementing data from different angles, helps to discover potential quality problems, and combines the results obtained by multiple detection methods to more reliably evaluate the quality of the weld and reduce the possibility of misjudgment. Among them, plane images can improve the details of the weld surface, such as surface defects such as cracks and pores, and stereoscopic scanning data can reveal the internal structure of the weld, such as internal defects such as lack of fusion and slag inclusions. Example 6

[0203] The present invention is a machine vision-based energy storage box weld appearance detection system, which is characterized by comprising a plane image acquisition detection module, a three-dimensional data scanning module, a stereo scanning data analysis module, a plane and three-dimensional fusion module and a visualization module;

[0204] The plane image acquisition and detection module is used to process the weld appearance image of the energy storage box to obtain the key features of the weld;

[0205] Among them, the key characteristics of the weld include the weld profile, width and color;

[0206] Process the key features of the weld to obtain the surface defects of the weld;

[0207] The three-dimensional data scanning module is used to obtain the three-dimensional scanning data of the weld and obtain the three-dimensional model of the weld;

[0208] The stereo scanning data analysis module obtains the defect area inside the weld based on the three-dimensional model of the weld;

[0209] The plane and three-dimensional fusion module is used to fuse the plane image and three-dimensional scanning data of the weld to generate a complete view of the weld;

[0210] The visualization module is used to display the complete 3D model of the weld.

[0211] Working principle of the invention: The technical solution of the invention relates to a method for integrated detection of weld surface and internal defects, which specifically includes the following steps: first, the appearance image of the weld is obtained by an image acquisition device, and then the contour, width, color and other features of the weld are extracted by an image processing algorithm; defects such as cracks and pits are identified based on these features; finally, the weld feature vector is calculated and compared with the standard normal weld feature vector to determine whether there are defects on the weld surface, and non-destructive testing technology (such as ultrasonic testing, X-ray testing, etc.) is used to obtain the internal image data of the weld and pre-process it; then three-dimensional reconstruction is performed to generate the weld. A three-dimensional model is obtained; defects inside the weld are identified by performing image processing on the three-dimensional model, and the weld appearance image obtained above is matched with characteristic points in the weld internal structure data obtained by stereo scanning technology; matching errors are calculated and necessary adjustments are made to ensure that the errors are within a predetermined acceptable range; the matched data are fused to generate a complete view including weld surface features and internal structure information. The technical solution of the present invention overcomes the problem of separation of weld surface and internal defect detection in the prior art, realizes integrated detection and comprehensive evaluation of weld surface features and internal structure information, thereby significantly improving the accuracy and reliability of weld detection.

[0212] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for detecting the appearance of a weld of an energy storage box based on machine vision, characterized in that: The following steps are involved: Obtain the weld appearance image of the energy storage box and obtain the key features of the weld; Among them, the key characteristics of the weld include the weld profile, width and color; Process the key features of the weld to obtain the surface defects of the weld; Acquire the stereoscopic scanning data of the weld and obtain the three-dimensional model of the weld; Based on the three-dimensional model of the weld, the defect area inside the weld is obtained; The process of obtaining the defect area inside the weld is: After the energy storage box completes the welding task, the image data inside the weld is obtained and recorded as three-dimensional scanning data; The three-dimensional scanning data is reconstructed in three dimensions to obtain a three-dimensional model of the weld; The three-dimensional model of the weld is converted into grayscale to obtain the grayscale value of each pixel in the three-dimensional model; All gray values ​​in the three-dimensional model of the weld are processed to obtain a histogram of the gray image; Perform threshold analysis on the histogram of the grayscale image to obtain the optimal threshold for image segmentation; Based on the optimal threshold, the three-dimensional model is segmented according to the threshold to obtain areas above the threshold and areas below the threshold; The area above the threshold is marked as a defective area and recorded as the defective area of ​​the weld; Fusion of the weld plane image and stereoscopic scanning data to generate a complete view of the weld; The process of fusing the planar image and stereoscopic scanning data of the weld is as follows: Based on the weld appearance image of the energy storage box, the starting point, the ending point and the corner point of the weld in the weld appearance image are obtained; The starting point, ending point and corner point of the weld are used as reference points and marked in the weld appearance image; Based on the reference points marked in the weld appearance image, the starting point, the end point and the corner point of the weld are marked as reference points in the stereo scan data; Obtain pixel coordinates of reference points in the weld appearance image; Obtain pixel coordinates of reference points in stereo scanning data; Based on the pixel coordinates of the reference point in the stereo scanning data, the starting point of the weld in the stereo scanning data is taken as the origin, the weld extension direction is the X axis, the direction perpendicular to the weld is the Y axis, and the direction perpendicular to the weld plane is the Z axis, to construct a three-dimensional coordinate system for the stereo scanning data; Based on the three-dimensional coordinate system of the stereoscopic scanning data and the pixel coordinates of the reference points in the weld appearance image, a coordinate conversion algorithm is used to convert the coordinates in the weld appearance image into the same three-dimensional coordinate system as the stereoscopic scanning data, which is recorded as the three-dimensional coordinate system of the appearance image; Processing the three-dimensional coordinate system of the appearance image to obtain the coordinate points of the weld contour and the coordinate points of the weld defects in the three-dimensional coordinate system of the appearance image, which are recorded as the appearance image feature points; Processing the three-dimensional coordinate system of the stereoscopic scanning data to obtain coordinate points of the weld contour and the weld defect in the three-dimensional coordinate system of the stereoscopic scanning data, which are recorded as stereoscopic scanning feature points; Match all appearance image feature points with corresponding stereo scanning feature points to obtain a three-dimensional coordinate system after the weld appearance image and the stereo scanning data are fused; Based on the three-dimensional coordinate system after the fusion of the weld appearance image and the stereoscopic scanning data, all the appearance image feature points and the corresponding stereoscopic scanning feature points are processed according to the Euclidean distance calculation formula to obtain the Euclidean distance between each appearance image feature point and the corresponding stereoscopic scanning feature point; Based on, through the formula The average distance error MDE between the appearance image feature points and the corresponding stereo scan feature points is calculated; Among them, D u is the distance between each pair of feature points, u is the number of feature points, u=1,2,…,N; Compare the mean distance error MDE with the mean distance error threshold; If the average distance error MDE is greater than the average distance error threshold, re-matching is performed; If the average distance error MDE is less than or equal to the average distance error threshold, a matching qualified signal is generated; Based on the matching qualified signals, the three-dimensional coordinate system of the stereoscopic scanning data is fused with the three-dimensional coordinate system of the appearance image to generate a complete view of the weld; Among them, the complete view of the weld includes the weld profile, surface defects, weld shape and weld view of internal defects.

2. The method for detecting the appearance of a weld of an energy storage box based on machine vision according to claim 1, characterized in that: After the energy storage box completes the welding task, obtain the weld appearance image of the energy storage box; Based on the weld appearance image of the energy storage box, position information of each pixel in the weld appearance image is obtained; Each pixel in the weld appearance image is mapped as a point into the plane coordinate system to obtain the coordinate position of each pixel, which is recorded as a pixel point; The weld appearance image and all pixel points in the weld appearance image are processed to obtain the complete outline of the weld.

3. The method for detecting the appearance of a weld of an energy storage box based on machine vision according to claim 2, characterized in that: Based on the complete contour of the weld, all pixel points (x i ,y i ) to obtain the minimum x value, maximum x value, minimum y value and maximum y value; Where i is the number of pixels on the weld contour, i=1,2,...,n; The maximum x value and the minimum x value are calculated as the difference to obtain the width of the weld.

4. The method for detecting the appearance of a weld of an energy storage box based on machine vision according to claim 3 is characterized in that: The process of obtaining the surface defects of the weld is: Obtaining a number of actual image samples representing a particular type of defect; Among them, specific types of defects include cracks and pits; The defective weld area in the actual image sample is cropped to obtain a template image; Process all template images to make the sizes of all template images consistent; Integrate all processed template images to obtain a defect template library; Based on the weld area in the energy storage box weld appearance image, the weld area is separated from the background in the energy storage box weld appearance image to obtain a separate weld area image; Compare the weld area image with one or more template images in the defect template library to obtain a correlation coefficient between the weld area image and the template image; comparing the correlation coefficient to a correlation coefficient threshold; If the correlation coefficient is greater than the correlation coefficient threshold, there is a defect in the template image at that position; For each location where the correlation coefficient is greater than the correlation coefficient threshold, the location is marked as a defective location.

5. The method for detecting the appearance of a weld of an energy storage box based on machine vision according to claim 4, characterized in that: The calculation process of the correlation coefficient between the weld area image and the template image is: Obtain the pixel value of each pixel in the weld area image; According to the mean calculation formula, the pixel value of each pixel in the weld area image is processed to obtain the average pixel value J of the weld area image. I ; Get the pixel value of each pixel in the template image; According to the mean calculation formula, the pixel value of each pixel in the template image is processed to obtain the average pixel value J of the template image. T ; By formula The correlation coefficient XGX is calculated; Among them, T is the template image, I is the weld area image, and I p (x, y) is the pixel value of the weld area image at position p, T(x, y) is the pixel value of the template image, ∑ x,y is the sum of all pixels of the template image, XGX is the sub-image I at a certain position p in the template image T and the weld area image I p The correlation coefficient between .

6. The method for detecting the appearance of a weld of an energy storage box based on machine vision according to claim 1, characterized in that: Surface defects of welds can also be obtained by: Based on the weld width and weld color features in the weld appearance image of the energy storage box; Among them, the color feature of the weld is the color histogram; The weld width and color histogram are combined to obtain a set of weld feature vectors; Obtain a set of feature vectors of normal welds; The characteristic vector of the weld is calculated by difference with the characteristic vector of a normal weld to obtain a characteristic vector deviation value; The squares of all characteristic vector deviation values ​​are summed up and the obtained value is squared to obtain the error value between the characteristic vector of the weld and the characteristic vector of the normal weld; comparing the error value to an error value threshold; If the error value is greater than the error value threshold, it means that there is a defect in the weld of the energy storage box.

7. A machine vision-based energy storage box weld appearance detection system, used to execute the machine vision-based energy storage box weld appearance detection method according to any one of claims 1 to 6, characterized in that: It includes a plane image acquisition and detection module, a three-dimensional data scanning module, a stereoscopic scanning data analysis module, a plane and three-dimensional fusion module and a visualization module; The plane image acquisition and detection module is used to obtain the weld appearance image of the energy storage box and process it to obtain the key features of the weld; Among them, the key characteristics of the weld include the weld profile, width and color; Process the key features of the weld to obtain the surface defects of the weld; The three-dimensional data scanning module is used to obtain the three-dimensional scanning data of the weld and obtain the three-dimensional model of the weld; The stereo scanning data analysis module obtains the defect area inside the weld based on the three-dimensional model of the weld; The plane and three-dimensional fusion module is used to fuse the plane image and three-dimensional scanning data of the weld to generate a complete view of the weld; The visualization module is used to display the complete three-dimensional model of the weld; The process of fusing the planar image and stereoscopic scanning data of the weld is as follows: Based on the weld appearance image of the energy storage box, the starting point, the ending point and the corner point of the weld in the weld appearance image are obtained; The starting point, ending point and corner point of the weld are used as reference points and marked in the weld appearance image; Based on the reference points marked in the weld appearance image, the starting point, the end point and the corner point of the weld are marked as reference points in the stereo scan data; Obtain pixel coordinates of reference points in the weld appearance image; Obtain pixel coordinates of reference points in stereo scanning data; Based on the pixel coordinates of the reference point in the stereo scanning data, the starting point of the weld in the stereo scanning data is taken as the origin, the weld extension direction is the X axis, the direction perpendicular to the weld is the Y axis, and the direction perpendicular to the weld plane is the Z axis, to construct a three-dimensional coordinate system for the stereo scanning data; Based on the three-dimensional coordinate system of the stereoscopic scanning data and the pixel coordinates of the reference points in the weld appearance image, a coordinate conversion algorithm is used to convert the coordinates in the weld appearance image into the same three-dimensional coordinate system as the stereoscopic scanning data, which is recorded as the three-dimensional coordinate system of the appearance image; Processing the three-dimensional coordinate system of the appearance image to obtain the coordinate points of the weld contour and the coordinate points of the weld defects in the three-dimensional coordinate system of the appearance image, which are recorded as the appearance image feature points; Processing the three-dimensional coordinate system of the stereoscopic scanning data to obtain coordinate points of the weld contour and the weld defect in the three-dimensional coordinate system of the stereoscopic scanning data, which are recorded as stereoscopic scanning feature points; Match all appearance image feature points with corresponding stereo scanning feature points to obtain a three-dimensional coordinate system after the weld appearance image and the stereo scanning data are fused; Based on the three-dimensional coordinate system after the fusion of the weld appearance image and the stereoscopic scanning data, all the appearance image feature points and the corresponding stereoscopic scanning feature points are processed according to the Euclidean distance calculation formula to obtain the Euclidean distance between each appearance image feature point and the corresponding stereoscopic scanning feature point; Based on, through the formula The average distance error MDE between the appearance image feature points and the corresponding stereo scan feature points is calculated; Among them, D u is the distance between each pair of feature points, u is the number of feature points, u=1,2,…,N; Compare the mean distance error MDE with the mean distance error threshold; If the average distance error MDE is greater than the average distance error threshold, re-matching is performed; If the average distance error MDE is less than or equal to the average distance error threshold, a matching qualified signal is generated; Based on the matching qualified signals, the three-dimensional coordinate system of the stereoscopic scanning data is fused with the three-dimensional coordinate system of the appearance image to generate a complete view of the weld; Among them, the complete view of the weld includes the weld profile, surface defects, weld shape and weld view of internal defects.

Images