Visual inspection method and system for welding quality of electric vehicle frame

Abstract

The invention relates to the technical field of quality detection and computer vision, and discloses a visual detection method and system for the welding quality of an electric vehicle frame, and the method comprises the steps: constructing a three-dimensional point cloud model of the frame; generating a welding seam geometric prior curve; a local segmentation algorithm driven by image gray statistics is adopted to obtain a welding seam coarse segmentation mask; fine segmentation and defect detection are carried out; and outputting a welding quality evaluation result. Compared with the prior art, the method for realizing weld joint judgment simply based on two-dimensional image gray level comparison or edge detection has the technical problem that accurate weld joint area extraction is difficult to realize particularly under the condition that the types of weld joints of an electric vehicle frame are complicated and the forms of butt welding, angle welding, T-shaped welding, lap welding and the like are different. Due to the fact that a registration mechanism of multi-view three-dimensional point cloud modeling and luminosity guiding is introduced, welding seam positioning and quality evaluation of a complex structure are achieved, and the automation level of electric vehicle frame welding quality detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of quality detection and computer vision, and particularly relates to a visual detection method and system for welding quality of an electric vehicle frame. BACKGROUND

[0002] At present, the welding quality is one of the key indicators of the safety and assembly reliability of the electric vehicle frame structure, and the subsequent service performance depends largely on the geometric consistency and continuity of the appearance of the weld. The traditional weld detection mainly relies on manual experience visual inspection, which is limited by the large difference in the level of inspectors, strong environmental light interference, detection fatigue accumulation and other factors, and has the problems of strong subjectivity, low efficiency and non-repeatability, which is difficult to meet the needs of automatic quality inspection of large quantities of frames.

[0003] In recent years, the welding appearance detection technology based on image recognition has been gradually popularized, the welding area image is obtained by an industrial camera, and the welding contour is identified and extracted by using a gray gradient, edge detection or deep learning segmentation network. Although this method can realize appearance defect detection with a certain accuracy under ideal conditions, there are still many deficiencies in actual frame detection: first, due to the complex structure of the frame member and the large difference in the welding form (such as butt welding, corner welding and lap welding), the existing image recognition algorithm generally lacks geometric shape prior constraint, which leads to easy identification deviation, fracture or background fusion of the welding edge under different viewing angles; second, in the scene where there is strong light reflection, insufficient penetration or member inclination in the welding area, the image gray scale distribution changes dramatically, and the algorithm based on two-dimensional pixel distribution for segmentation has poor robustness and is easy to misidentify the non-weld structure such as weld shadow and splash residue. The existing technology cannot fully meet the needs of automatic extraction, geometric recovery and quality quantitative evaluation of multiple types of welds on complex electric vehicle frames under the conditions of multiple viewing angles, unstable lighting, lack of fixture positioning and the like.

[0004] Therefore, it is urgent to propose a visual detection method for welding quality of an electric vehicle frame, which can still realize the extraction of welding structure continuity and intelligent judgment of appearance geometric quality under the conditions of irregular welding imaging, complex welding configuration and other industrial actual conditions, so as to improve the detection accuracy and support efficient closed-loop quality inspection deployment of automatic production lines. SUMMARY

[0005] In view of the above technical deficiencies, the purpose of the present application is to propose a visual detection method for welding quality of an electric vehicle frame, which aims to solve the technical problem that the existing method based on simple two-dimensional image gray scale contrast or edge detection for welding identification is difficult to realize accurate welding area extraction, especially under the condition that the welding types of the electric vehicle frame are various, including butt welding, corner welding, T-shaped welding and lap welding with different forms.

[0006] To solve the above technical problems, the present application adopts the following technical solutions: the present application provides a visual detection method for welding quality of an electric vehicle frame, The visual detection method for welding quality of the electric vehicle frame comprises: Step S10: acquire a multi-view image sequence set I and a laser stripe image set L of the frame to be detected, and construct a frame three-dimensional point cloud data set P based on the multi-view image sequence set I and the laser stripe image set L by using a multi-view photometric constraint-based laser stripe extraction mechanism; Step S20: acquire a welding component axis vector set V of the frame to be detected, and perform a frame structure skeleton extraction operation based on the frame three-dimensional point cloud data set P and the welding component axis vector set V by using a projection adaptive registration mechanism, and output a weld geometry prior curve set; Step S30: perform a weld rough segmentation processing according to the weld geometry prior curve set by using an adaptive local segmentation mechanism based on image gray scale statistics driving, and output a weld rough segmentation mask M; Step S40: perform a weld fine segmentation processing based on the weld rough segmentation mask M by using a skeleton alignment optimization mechanism based on morphological guidance and geometric consistency constraint, and output a weld fine segmentation result set; Step S50: calculate a weld appearance geometry index based on the weld fine segmentation result set by using a breakpoint connectivity analysis method, wherein the weld appearance geometry index comprises a weld excess height, a weld width and a weld continuity score; and output a weld quality judgment report of the frame to be detected according to the weld appearance geometry index in combination with a preset quality judgment rule.

[0007] Preferably, in step S10, the steps of acquiring the multi-view image sequence set I and the laser stripe image set L of the frame to be detected, and constructing the frame three-dimensional point cloud data set P based on the multi-view image sequence set I and the laser stripe image set L by using the multi-view photometric constraint-based laser stripe extraction mechanism, specifically comprise: Step S101: dispose a fixed industrial camera and a structured light projection device in a detection area of the frame to be detected, and acquire the multi-view image sequence set I and the laser stripe image set L of the frame to be detected by the fixed industrial camera and the structured light projection device; Step S102: acquire a weld area of the frame to be detected, and determine a stripe offset amount Δx of the laser stripe generated in the weld area of the frame to be detected based on the multi-view image sequence set I and the laser stripe image set L by using the multi-view photometric constraint-based laser stripe extraction mechanism; Step S103: calculate a three-dimensional coordinate depth map of the weld area by triangulation method according to the stripe offset amount Δx and preset camera calibration parameters, and construct the frame three-dimensional point cloud data set P according to the three-dimensional coordinate depth map.

[0008] Preferably, step S10, which involves determining the stripe offset Δx generated by the laser stripes in the weld area of ​​the vehicle frame under inspection using a laser stripe extraction mechanism based on multi-view photometric constraints, based on the multi-view image sequence set I and the laser stripe image set L, specifically includes: Based on the preset frame design model, the region of interest (ROI) of the weld is extracted from the multi-view image sequence set I, and a multi-view collaborative stripe sub-image set is extracted from the laser stripe image set L according to the ROI of the weld. Obtain the scan line i along the transverse tangent direction of the laser stripe in the multi-view collaborative stripe sub-image set, and extract the brightness center of the scan line. According to the center of brightness Photometric consistency residuals are constructed using the least squares method based on multi-view photometric consistency constraints. And based on photometric consistency residuals Define the photometric residual confidence function When the photometric residual confidence function When the value is less than 0.5, it is determined that there is high reflectivity, obstruction or splash contamination, and the scan line i in the transverse tangent direction of the laser stripe is subjected to stripe fitting and shielding processing, and the effective stripe scan line set is output. For the output set of valid stripe scan lines, a third-order parabola is used to perform sub-pixel stripe center fitting, and the stripe offset Δx is output.

[0009] Preferably, in step S20, the steps of obtaining the set of axis vectors V of the welded components of the frame to be inspected, performing the frame structure skeleton extraction operation based on the frame 3D point cloud data set P and the set of axis vectors V of the welded components using a projection adaptive registration mechanism, and outputting the set of prior geometric curves of the weld seam specifically include: Step S201: Obtain the set of axis vectors V of the welded components of the frame to be inspected. For each point cloud data p in the frame 3D point cloud data set P, calculate the distance from point cloud data p to the corresponding component axis in the set of axis vectors V of the welded components. Minimum projection distance Based on minimum projection distance Define a Gaussian weighting function, and select the point cloud data with the largest Gaussian weighting function from the 3D point cloud data set P of the vehicle frame as the component axis. The corresponding point cloud is clustered, and the point cloud cluster set C is output; Step S202: Perform geometric topology analysis using multi-scale connectivity graph analysis based on the point cloud cluster set C, and construct a candidate set of weld connection points based on the results of the geometric topology analysis; Step S203: Calculate the angle of the connecting axis direction using the cosine angle analysis method based on the candidate set of weld connection points; when the angle of the connecting axis direction is less than or equal to 90°, perform the fitting operation using B-spline curves or circular arc curves; when the angle of the connecting axis direction is greater than 90°, perform the fitting operation using the weighted least squares fitting method; finally, output the set of prior geometric curves of the weld based on the fitting results.

[0010] Preferably, in step S20, the set of weld geometry prior curves is used to represent the weld types between intersecting components in the CAD model corresponding to the frame to be inspected. The weld types between intersecting components include butt welds, fillet welds, T-welds, and lap welds.

[0011] Preferably, step S30, which involves performing coarse weld segmentation based on an adaptive local segmentation mechanism driven by image grayscale statistics according to the weld geometric prior curve set, and outputting the coarse weld segmentation mask M, specifically includes: Step S301: Extract the point cloud image sub-block S corresponding to the weld geometry prior curve set from the 3D point cloud data set P; Step S302: Calculate the local gray mean and local gray standard deviation for the point cloud image sub-block S, and generate a gray threshold map based on the local gray mean and local gray standard deviation using the Sauvola adaptive threshold segmentation algorithm. Step S303: Perform preliminary binary segmentation based on the grayscale threshold image to output the weld coarse segmentation mask M.

[0012] Preferably, step S40, which involves performing fine weld segmentation based on the coarse weld segmentation mask M using a skeleton alignment optimization mechanism based on shape guidance and geometric consistency constraints, and outputting a set of fine weld segmentation results, specifically includes: Step S401: Extract connected regions from the weld coarse segmentation mask M, calculate the weld centerline sketch using the distance transformation method for the connected regions, and extract the initial skeleton curve segment based on the weld centerline sketch using the morphological skeleton extraction algorithm. Step S402: Introduce the first constraint energy Second constraint energy Based on the initial skeleton curve segments and the prior geometric curve set of the weld, combined with the first constraint energy... Second constraint energy Perform geometric consistency alignment matching to obtain the aligned skeleton curve set; where the first constraint energy... The second constraint energy is used to constrain the optimized skeleton curve to maintain consistency with the morphological structure of the coarsely segmented region. Used to constrain the skeleton curve and the set of prior geometric curves of the weld to maintain the continuity of the geometric shape and the consistency of the overall trend; Step S403: Using the aligned skeleton curve set as the center, and combining it with the preset skeleton point width adjustment window, reconstruct to form a set of fine weld segmentation results.

[0013] The present invention also provides a visual inspection system for the welding quality of electric vehicle frames, comprising: The 3D reconstruction module is used to acquire a set of multi-view image sequences I and a set of laser stripe images L of the chassis to be inspected. Based on the set of multi-view image sequences I and the set of laser stripe images L, a laser stripe extraction mechanism based on multi-view photometric constraints is used to construct a 3D point cloud data set P of the chassis. The weld prior modeling module is used to obtain the set of axis vectors V of the welded components of the frame to be inspected. Based on the frame 3D point cloud data set P and the set of axis vectors V of the welded components, the frame structure skeleton extraction operation is performed using a projection adaptive registration mechanism, and the set of weld geometric prior curves is output. The weld coarse segmentation module is used to perform weld coarse segmentation processing based on an adaptive local segmentation mechanism driven by image grayscale statistics, according to the set of weld geometric prior curves, and outputs a weld coarse segmentation mask M. The skeleton alignment optimization module is used to perform fine weld segmentation based on the skeleton alignment optimization mechanism based on shape guidance and geometric consistency constraints, based on the coarse weld segmentation mask M, and outputs a set of fine weld segmentation results. The appearance evaluation and judgment module is used to calculate the weld appearance geometric indicators based on the weld fine segmentation result set using the breakpoint connectivity analysis method. The weld appearance geometric indicators include weld reinforcement height, weld width, and weld continuity score. Based on the weld appearance geometric indicators and the preset quality judgment rules, the module outputs a weld quality judgment report for the vehicle frame to be inspected.

[0014] The present invention also provides a visual inspection device for the welding quality of electric vehicle frames, comprising: a memory, a processor, and a visual inspection program for the welding quality of electric vehicle frames stored in the memory and executable on the processor. When the visual inspection program for the welding quality of electric vehicle frames is executed by the processor, a visual inspection method for the welding quality of electric vehicle frames is implemented.

[0015] The present invention also provides a computer program product, including a visual inspection program for the welding quality of an electric vehicle frame, wherein the visual inspection program for the welding quality of the electric vehicle frame, when executed by a processor, implements the visual inspection method for the welding quality of the electric vehicle frame.

[0016] The beneficial effects of this invention are as follows: This invention proposes a three-dimensional reconstruction method based on multi-view image recognition and laser stripe fusion, which effectively solves the problem of inaccurate weld contour extraction in traditional two-dimensional visual inspection under the conditions of occlusion and lighting interference, and realizes high-precision modeling of complex welds on vehicle frames.

[0017] This invention employs an adaptive registration and local segmentation mechanism that integrates the prior geometric curves of the weld seam, combined with morphological guidance and geometric consistency optimization strategies. It can adapt to the quality inspection needs of different electric vehicle frames under different welding conditions (such as weld seam offset, local deformation, etc.). Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of a visual inspection method for the welding quality of an electric vehicle frame according to the present invention.

[0020] Figure 2 This is a schematic diagram showing the weld seam recognition effect of a first embodiment of a visual inspection method for the welding quality of an electric vehicle frame according to the present invention.

[0021] Figure 3 This is a schematic diagram comparing the effects of weld seam identification skeleton alignment optimization technology in the first embodiment of the visual inspection method for welding quality of electric vehicle frame of the present invention.

[0022] Figure 4 This is a schematic diagram of the equipment for a visual inspection method of welding quality of an electric vehicle frame according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1: As Figure 1 The diagram shown is a flowchart of the first embodiment of the visual inspection method for the welding quality of electric vehicle frames according to the present invention, which presents the first embodiment of the visual inspection method for the welding quality of electric vehicle frames according to the present invention.

[0025] In the first embodiment, the visual inspection method for the welding quality of the electric vehicle frame includes: Step S10: Obtain the multi-view image sequence set I and the laser stripe image set L of the frame to be inspected. Based on the multi-view image sequence set I and the laser stripe image set L, construct the frame three-dimensional point cloud data set P using a laser stripe extraction mechanism based on multi-view photometric constraints. It should be noted that the laser stripe extraction mechanism based on multi-view photometric constraints refers to simultaneously acquiring laser stripe images containing the weld seam area using multiple industrial cameras with fixed viewpoints. By establishing photometric consistency constraints under different viewpoints, the light intensity response of the laser stripes in the multi-view space is jointly modeled and matched, thereby achieving high-precision extraction of laser stripe pixels. This mechanism includes three main steps: first, photometric normalization processing of multi-view images to eliminate differences in ambient lighting; second, photometric residual calculation based on pixel matching to filter stripe pixels that meet the brightness consistency condition; and third, 3D reconstruction based on a triangulation model to project effective stripe points onto a unified 3D coordinate system, forming a high-precision point cloud dataset P. The multi-view photometric constraints mentioned in this step not only include brightness matching between images but also structural similarity matching and grayscale gradient consistency judgment, thus enabling stable extraction of the true laser reflection contour of the weld seam area in complex scenarios with reflections, occlusions, or weld spatter impurities.

[0026] Understandably, this step, by jointly identifying multi-view images and laser stripe information, can significantly improve the stripe recognition accuracy and spatial consistency of the weld area. Compared to recognition methods that rely solely on a single camera viewpoint, this mechanism can compensate for recognition by utilizing effective information from other viewpoints, even when the weld has strong reflections, uneven surfaces, or occlusions, thereby achieving complete stripe extraction of the weld area. The introduction of the multi-view photometric constraint mechanism means that point cloud reconstruction no longer relies on the stability of a single frame image, but instead obtains stripe point confidence through cross-view photometric residual statistics, thus forming a continuous and high-confidence three-dimensional surface of the weld in space. Therefore, this step of the present invention achieves deep fusion of image recognition and laser ranging under weld imaging conditions, improving point cloud density, reducing noise, and providing high-quality input data for subsequent geometric modeling and weld quality analysis.

[0027] It should be understood that, compared to traditional weld seam recognition technologies based on single-view or dual-view stereo matching, the multi-view photometric constraint mechanism introduced in this step has significant advantages in the complex weld seam scenarios of electric vehicle frames. Traditional solutions often assume that the weld seam surface is a diffuse reflection model, ignoring the specular reflection characteristics of highly reflective metal surfaces, resulting in significant stripe drift and large reconstruction errors in highly reflective areas. This invention, by establishing photometric consistency constraints from multiple perspectives, can automatically detect and eliminate reflective interference areas, excluding pixels that do not meet photometric consistency requirements from stripe matching, thereby effectively reducing 3D reconstruction errors caused by material reflection or surface unevenness.

[0028] For example, such as Figure 2 As shown, in the complex scenario of inspecting weld seams on electric vehicle frames, traditional solutions typically employ stereo matching methods based on single or dual perspectives to identify and reconstruct laser stripes in 3D. However, due to the strong specular reflection characteristics of the frame surface, bright areas are easily formed under strong light, causing stripe matching to drift or break, resulting in serious errors in the reconstructed weld morphology. Figure 2 The left side shows that the weld identification results of the traditional method show obvious discontinuities in the reflective area, failing to guarantee the continuity and accuracy of weld extraction. However, Figure 2 The right side shows the multi-view image photometric consistency constraint mechanism introduced in this invention. By detecting areas that do not meet photometric consistency under different viewpoints and automatically removing the corresponding pixels, the interference of reflective areas on laser stripe recognition is effectively avoided, making the weld seam extraction more complete and smooth visually.

[0029] Step S20: Obtain the set of axis vectors V of the welded components of the frame to be inspected. Based on the frame 3D point cloud data set P and the set of axis vectors V of the welded components, perform the frame structure skeleton extraction operation using the projection adaptive registration mechanism, and output the set of prior geometric curves of the weld. It should be noted that the "set of welded component axis vectors V" refers to a set of three-dimensional direction vectors constructed based on the known design reference axes of each welded component in the CAD model, process drawings, or laser scanning data of the electric vehicle frame. This set is used to identify the theoretical installation direction and spatial position relationship of each welded component. The "projection adaptive registration mechanism" refers to dynamically constructing a point cloud filtering function based on the geometric constraint relationship between the frame's three-dimensional point cloud data P and V, and combining it with the weighted minimum distance criterion to achieve adaptive alignment and registration of point cloud clusters towards the component axes, thereby extracting the spatial skeleton path along the weld direction. Finally, the extracted set of weld geometric prior curves provides a structural reference benchmark for subsequent weld area positioning, segmentation, and determination.

[0030] Understandably, this step, by combining the 3D point cloud data P with the set of axis vectors V of the welded components, can effectively extract point cloud clusters with spatial continuity and structural consistency within the weld area without relying on fitting a complete CAD model. This is achieved by calculating the minimum projection distance from each point to the axis of each component and constructing local point cloud clusters using Gaussian weighting.

[0031] It should be understood that, compared with traditional methods of skeleton extraction using fixed axial envelope boxes or boundary volume enclosing methods, the "projection adaptive registration mechanism" proposed in this step establishes a linkage between spatial orientation priors and point cloud spatial projection distances. This allows for dynamic adjustment of the aggregation area of ​​local point clouds based on actual weld seam direction changes, and automatic formation of skeleton paths guided by component axes. Thus, even in cases of complex structural component intersections, large weld seam corner changes, or incomplete local point clouds, the geometric continuity and structural correctness of the extracted skeleton can still be maintained.

[0032] For example, regarding the front fork area at the bottom of an electric vehicle frame, the presence of a diagonal weld between the fork and the chassis beam, coupled with reflections and occlusions in this area during laser acquisition, makes traditional skeleton recognition methods based on fixed ROI or boundary extraction prone to losing connection segment information and generating broken weld paths. This step employs a projection adaptive registration mechanism, using the theoretical axes of the fork and chassis beam as the prior vector V, combined with the density distribution of the point cloud P. This enables the formation of a continuous and structurally logical weld prior curve in the diagonal transition segment, exhibiting stronger structural recovery capabilities, especially in connection segments with an angle greater than 60°.

[0033] Step S30: Based on the set of prior geometric curves of the weld, perform coarse segmentation of the weld using an adaptive local segmentation mechanism driven by image grayscale statistics, and output the coarse segmentation mask M of the weld. It should be noted that the "adaptive local segmentation mechanism driven by image grayscale statistics" refers to the following: for the local region indicated by the set of prior geometric curves of the weld seam, a series of regions of interest (ROIs) are defined near each prior curve, and grayscale histogram features (such as mean, variance, skewness, kurtosis, etc.) are independently statistically analyzed within each ROI. An initial segmentation mask is generated by combining an adaptive thresholding strategy (such as Otsu-based or local dynamic histogram thresholding).

[0034] Understandably, this step, by fusing geometric prior guidance with adaptive modeling of local grayscale features, can significantly reduce the false detection problems common in traditional global image segmentation methods. Especially when the weld seam has discontinuous, occluded, or strongly reflective areas, it can still accurately preserve the main area of ​​the weld seam. The adaptive segmentation mechanism not only improves the accuracy of weld seam contour capture but also provides a high-confidence initial region for subsequent fine skeleton fitting, effectively improving the complete extraction rate of the entire weld seam.

[0035] It should be understood that, compared with the background leakage and weld fracture problems caused by the use of fixed thresholds or full image equalization in traditional methods, this invention effectively suppresses the uneven illumination and metal reflection interference caused by the frame material by locally modeling the image grayscale distribution under the constraint of geometric prior curves and combining it with an adaptively optimized grayscale distribution statistical method, thereby enhancing the separability of the weld area in the image.

[0036] Step S40: Based on the coarse weld segmentation mask M, a skeleton alignment optimization mechanism based on shape guidance and geometric consistency constraints is used to perform fine weld segmentation processing, and output a set of fine weld segmentation results. It should be noted that the "skeleton alignment optimization mechanism" refers to using the morphological and structural features in the coarse segmentation area of ​​the weld as the basis for initial skeleton extraction, while introducing geometric prior curves to optimize the alignment, thereby achieving accurate positioning and edge reconstruction of the weld boundary.

[0037] Understandably, this step integrates "pixel-level morphological information in the image" with "structural prior information extracted from the 3D point cloud" to form a top-down fine segmentation process. This process not only preserves the accuracy of edge perception based on the original image grayscale information, but also uses geometric priors to suppress pseudo-segmentation regions caused by background noise, lighting changes, or workpiece boundaries, thereby improving the continuity and shape matching of segmentation edges.

[0038] It should be understood that, compared to traditional image segmentation algorithms that rely solely on local image texture or pixel brightness, problems such as skeleton breakage, boundary offset, or failure to identify cold weld areas often occur in complex vehicle frame scenarios with non-linear weld seam orientations. However, this step guides skeleton alignment through geometric consistency, which can accurately track weld seam orientation even in backgrounds with strong reflection interference, and repairs incomplete areas caused by occlusion or noise in the coarse segmentation stage, thereby effectively improving the accuracy of fine segmentation.

[0039] For example, such as Figure 3As shown, traditional skeleton extraction methods are prone to jagged shapes and fracture distortions when the weld grayscale distribution is complex or the background noise interference is strong, failing to accurately reflect the true direction and structure of the weld. The skeleton alignment optimization mechanism introduced in this invention, based on morphological guidance and geometric consistency constraints, can achieve smoothing and alignment of the skeleton shape while preserving the weld structural morphological features. This is achieved through continuity constraints matching the initial skeleton with the geometric prior curve, significantly improving the stability and accuracy of weld centerline extraction. This provides a foundation for the accurate evaluation of subsequent geometric parameters such as weld width and reinforcement height.

[0040] Step S50: Based on the weld fine segmentation result set, the weld appearance geometric index is calculated using the breakpoint connectivity analysis method. The weld appearance geometric index includes weld reinforcement height, weld width, and weld continuity score. Based on the weld appearance geometric index and the preset quality judgment rules, the weld quality judgment report of the frame to be inspected is output.

[0041] It should be noted that the breakpoint connectivity analysis method refers to identifying the locations and distribution characteristics of breakpoints in weld segments by using pixel-level connectivity judgment, jump length threshold filtering, and skeleton path tracking to extract the weld region contour from the fine weld segmentation result set. Based on this, a weld width extraction algorithm is used to statistically analyze the cross-sectional width at different locations, a region height mapping algorithm is used to calculate the weld reinforcement height, and the weld continuity score is calculated by combining the breakpoint spacing and distribution density. The final weld appearance geometric indicators include three categories: weld width, representing the lateral extension along the weld skeleton direction; weld reinforcement height, representing the average convex height of the weld region relative to the surrounding substrate surface; and weld continuity score, representing the overall connectivity ratio of the weld and reflecting the stability of weld quality.

[0042] Understandably, this step achieves the quantitative conversion of weld quality from image segmentation results to engineering geometric indicators. Through the breakpoint connectivity analysis method, it can not only identify local defects or fracture areas in the weld, but also assess their severity based on spatial statistical distribution, thereby forming a visualized and structured basis for quality judgment and effectively improving the objectivity and consistency of quality inspection.

[0043] Example 2: Furthermore, the present invention provides a visual inspection system for the welding quality of electric vehicle frames, employing a visual inspection method for the welding quality of electric vehicle frames described in the above embodiments, which can solve the technical problem of visual inspection of the welding quality of electric vehicle frames. Compared with the prior art, the beneficial effects of the visual inspection system for the welding quality of electric vehicle frames provided by the present invention are the same as those of the visual inspection method for the welding quality of electric vehicle frames provided in the above embodiments, and other technical features of the visual inspection system for the welding quality of electric vehicle frames are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0044] Example 3: This invention provides a visual inspection device for the welding quality of electric vehicle frames. Please refer to... Figure 4A visual inspection device for the welding quality of an electric vehicle frame includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the visual inspection method for the welding quality of an electric vehicle frame as described in Embodiment 1 above. The visual inspection device for the welding quality of an electric vehicle frame in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. This visual inspection device for the welding quality of an electric vehicle frame is merely an example and should not limit the functionality or scope of use of the embodiments of this invention. The visual inspection device for the welding quality of an electric vehicle frame may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. Random access memory 1004 also stores various programs and data required for the operation of a visual inspection device for the welding quality of an electric vehicle frame. Processing device 1001, read-only memory 1002, and random access memory 1004 are interconnected via bus 1005. I / O interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the visual inspection device for the welding quality of an electric vehicle frame to communicate wirelessly or wiredly with other devices to exchange data. Although a visual inspection device for the welding quality of an electric vehicle frame with various systems is shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0045] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the visual inspection method for the welding quality of an electric vehicle frame as described above. The computer program product provided by this invention can solve the technical problem of visual inspection of the welding quality of an electric vehicle frame. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as the beneficial effects of the visual inspection method for the welding quality of an electric vehicle frame provided in the above embodiments, and will not be repeated here.

[0046] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0047] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0048] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for visually inspecting the welding quality of an electric vehicle frame, characterized in that, The method comprises: Step S10: obtaining a multi-view image sequence set I and a laser stripe image set L of the vehicle frame to be detected, and constructing a vehicle frame three-dimensional point cloud data set P based on the multi-view image sequence set I and the laser stripe image set L by using a multi-view photometric constraint-based laser stripe extraction mechanism; Step S20: obtaining a welding component axis vector set V of the vehicle frame to be detected, and performing a vehicle frame structure skeleton extraction operation based on the vehicle frame three-dimensional point cloud data set P and the welding component axis vector set V by using a projection adaptive registration mechanism, and outputting a welding seam geometric prior curve set; Step S30: performing welding seam rough segmentation processing according to the welding seam geometric prior curve set by using an adaptive local segmentation mechanism based on image gray scale statistics driving, and outputting a welding seam rough segmentation mask M; Step S40: performing welding seam fine segmentation processing based on the welding seam rough segmentation mask M by using a skeleton alignment optimization mechanism based on morphological guidance and geometric consistency constraint, and outputting a welding seam fine segmentation result set; Step S50: calculating welding seam appearance geometric indexes based on the welding seam fine segmentation result set by using a breakpoint connectivity analysis method, wherein the welding seam appearance geometric indexes comprise welding seam excess height, welding seam width and welding seam continuity score; and outputting a welding seam quality judgment report of the vehicle frame to be detected according to the welding seam appearance geometric indexes and a preset quality judgment rule.

2. The method of claim 1, wherein the method further comprises: In step S10, the multi-view image sequence set I and the laser stripe image set L of the vehicle frame to be detected are obtained, and the vehicle frame three-dimensional point cloud data set P is constructed based on the multi-view image sequence set I and the laser stripe image set L by using the multi-view photometric constraint-based laser stripe extraction mechanism, and the steps specifically comprise: Step S101: arranging a fixed industrial camera and a structured light projection device in a detection area of the vehicle frame to be detected, and obtaining the multi-view image sequence set I and the laser stripe image set L of the vehicle frame to be detected by using the fixed industrial camera and the structured light projection device; Step S102: obtaining a welding seam area of the vehicle frame to be detected, and determining a stripe offset Δx of the laser stripe generated in the welding seam area of the vehicle frame to be detected based on the multi-view image sequence set I and the laser stripe image set L by using the multi-view photometric constraint-based laser stripe extraction mechanism; Step S103: calculating a three-dimensional coordinate depth map of the welding seam area by using a triangulation method according to the stripe offset Δx and preset camera calibration parameters, and constructing the vehicle frame three-dimensional point cloud data set P according to the three-dimensional coordinate depth map.

3. The method of claim 2, wherein the method further comprises: In step S10, the multi-view image sequence set I and the laser stripe image set L are obtained, and the stripe offset Δx of the laser stripe generated in the welding seam area of the vehicle frame to be detected is determined based on the multi-view image sequence set I and the laser stripe image set L by using the multi-view photometric constraint-based laser stripe extraction mechanism, and the steps specifically comprise: extracting a welding seam region of interest ROI from the multi-view image sequence set I based on a preset vehicle frame design model, and obtaining a multi-view collaborative stripe sub-image set from the laser stripe image set L according to the welding seam region of interest ROI; Obtaining a scanning line i in a laser stripe transverse tangent direction in a multi-view cooperative stripe sub-image set, extracting a brightness center of the scanning line , judging whether the brightness center is located in a valid region according to a valid region of the scanning line , and constructing a photometric consistency residual based on a least square method based on a multi-view photometric consistency constraint , and defining a photometric residual confidence function based on the photometric consistency residual ; when the photometric residual confidence function is less than 0.5, judging that there is high reflection, shielding or splashing pollution, performing stripe fitting shielding processing on the scanning line i in the laser stripe transverse tangent direction, and outputting a valid stripe scanning line set performing sub-pixel stripe center fitting on the output effective stripe scan line set by using a third-order parabola, and outputting the stripe offset Δx.

4. The method of claim 1, wherein the method further comprises: In step S20, the welding member axis vector set V of the to-be-inspected vehicle frame is obtained, and a projection adaptive registration mechanism is adopted based on the vehicle frame three-dimensional point cloud data set P and the welding member axis vector set V to perform a vehicle frame structure skeleton extraction operation, and a step of outputting a welding seam geometric prior curve set, specifically comprising: Step S201: Obtain the set of axis vectors V of the welded components of the frame to be inspected. For each point cloud data p in the frame 3D point cloud data set P, calculate the distance from point cloud data p to the corresponding component axis in the set of axis vectors V of the welded components. Minimum projection distance Based on minimum projection distance Define a Gaussian weighting function, and select the point cloud data with the largest Gaussian weighting function from the 3D point cloud data set P of the vehicle frame as the component axis. The corresponding point cloud is clustered, and the point cloud cluster set C is output; In step S202, a multi-scale connected graph analysis method is adopted based on the point cloud clustering set C to perform geometric topology analysis, and a welding seam connection point candidate set is constructed according to the result of the geometric topology analysis; In step S203, a cosine angle analysis method is adopted to calculate the connection axis direction angle according to the welding seam connection point candidate set; when the connection axis direction angle is less than or equal to 90°, a B-spline curve or a circular arc curve is adopted to perform fitting operation; when the connection axis direction angle is greater than 90°, a weighted least squares fitting method is adopted to perform fitting operation; finally, the welding seam geometric prior curve set is output according to the fitting result.

5. The method of claim 1, wherein the method further comprises: In step S20, the welding seam geometric prior curve set is used to represent the welding seam type between the intersecting members in the corresponding CAD model of the to-be-inspected vehicle frame, and the welding seam type between the intersecting members includes butt welding, fillet welding, T-type welding and lap welding.

6. The method of claim 1, wherein the method further comprises: In step S30, an adaptive local segmentation mechanism based on image gray scale statistical driving is adopted based on the welding seam geometric prior curve set to perform welding seam rough segmentation processing, and a step of outputting a welding seam rough segmentation mask M, specifically comprising: In step S301, a point cloud image sub-block S corresponding to the welding seam geometric prior curve set is extracted from the three-dimensional point cloud data set P; In step S302, local gray mean and local gray standard deviation are calculated for the point cloud image sub-block S, and a Sauvola adaptive threshold segmentation algorithm is adopted to generate a gray threshold map according to the local gray mean and the local gray standard deviation; In step S303, a preliminary binary segmentation operation is performed based on the gray threshold map, and the welding seam rough segmentation mask M is output.

7. The visual inspection method for the welding quality of an electric vehicle frame as described in claim 1, characterized in that, In step S40, a skeleton alignment optimization mechanism based on morphological guidance and geometric consistency constraint is adopted based on the welding seam rough segmentation mask M to perform welding seam fine segmentation processing, and a step of outputting a welding seam fine segmentation result set, specifically comprising: In step S401, a connected region is extracted from the welding seam rough segmentation mask M, and a welding seam centerline sketch is calculated for the connected region by using a distance transformation method, and an initial skeleton curve segment is extracted by using a morphological skeleton extraction algorithm according to the welding seam centerline sketch; Step S402: introducing a first constraint energy and a second constraint energy ; based on the initial skeleton curve segments and the set of weld geometry prior curves, combining the first constraint energy and the second constraint energy performing geometric consistency alignment matching to obtain an aligned skeleton curve set; wherein the first constraint energy is used to constrain the optimized skeleton curve to maintain consistency with the morphological structure of the rough segmentation region; the second constraint energy is used to constrain the skeleton curve to maintain geometric shape continuity and overall trend consistency with the set of weld geometry prior curves; In step S403, the aligned skeleton curve set is taken as the center to combine a preset skeleton point width adjustment window to form a welding seam fine segmentation result set. 8.A visual inspection system for welding quality of an electric vehicle frame, applied to the visual inspection method for welding quality of an electric vehicle frame in any one of claims 1 to 7, characterized in that, The visual detection system for the welding quality of the electric vehicle frame comprises: A three-dimensional reconstruction module is configured to obtain a multi-view image sequence set I and a laser stripe image set L of a to-be-inspected vehicle frame, and construct a vehicle frame three-dimensional point cloud data set P based on the multi-view image sequence set I and the laser stripe image set L by using a laser stripe extraction mechanism based on multi-view photometric constraint; A welding seam prior modeling module is configured to obtain a welding member axis vector set V of the to-be-inspected vehicle frame, and perform a vehicle frame structure skeleton extraction operation based on the vehicle frame three-dimensional point cloud data set P and the welding member axis vector set V by using a projection adaptive registration mechanism, and output a welding seam geometric prior curve set; The weld rough segmentation module is configured to perform a rough segmentation of the weld based on a set of weld geometric prior curves and an adaptive local segmentation mechanism driven by image gray scale statistics, and output a weld rough segmentation mask M; The skeleton alignment optimization module is configured to perform a fine segmentation of the weld based on the weld rough segmentation mask M and a skeleton alignment optimization mechanism based on morphological guidance and geometric consistency constraints, and output a set of weld fine segmentation results; The appearance evaluation and determination module is configured to calculate weld appearance geometric indicators based on the set of weld fine segmentation results and a breakpoint connectivity analysis method, wherein the weld appearance geometric indicators include weld reinforcement, weld width, and weld continuity score; and output a weld quality determination report of the to-be-inspected vehicle frame based on the weld appearance geometric indicators and a preset quality determination rule.

9. A visual inspection apparatus for detecting welding quality of an electric vehicle frame, characterized by, The visual detection device for the welding quality of the electric vehicle frame comprises a memory, a processor, and a visual detection program for the welding quality of the electric vehicle frame stored in the memory and executable on the processor, and the visual detection program for the welding quality of the electric vehicle frame, when executed by the processor, implements the visual detection method for the welding quality of the electric vehicle frame according to any one of claims 1 to 7.

10. A computer program product, characterised in that, The computer program product comprises a visual detection program for the welding quality of the electric vehicle frame, and the visual detection program for the welding quality of the electric vehicle frame, when executed by the processor, implements the visual detection method for the welding quality of the electric vehicle frame according to any one of claims 1 to 7.

Images