A weld surface detection system and method based on machine vision

Abstract

The present application provides a kind of welding surface detection system and method based on machine vision, the welding operation image of workpiece is multi-target visual identification, obtains the solder form change information of workpiece welding surface, to determine the solder filling state information, the filling space of solder on workpiece welding surface and compact type are comprehensively identified;Based on the solder filling state information, determine the sub-region of workpiece welding surface that needs secondary welding, and the secondary welding operation image of sub-region is visually identified, obtains the flow state information of solder in sub-region, the filling flow of solder in secondary welding operation process is identified in space, accurately determine the abnormal position of solder filling in workpiece welding surface;Again based on the distribution of solder filling abnormal position of workpiece welding surface, adjust the welding action state of welding tool to workpiece welding surface, accurately adjust the flow filling condition of solder in workpiece welding surface, effectively fill the air gap defect of workpiece welding surface, improve the welding quality of workpiece welding surface.

Description

Technical Field

[0001] This invention relates to the field of machine vision recognition, and more particularly to a machine vision-based weld surface inspection system and method. Background Technology

[0002] Welding is a common method in the assembly and fabrication of mechanical parts. Solder is heated to a high temperature and liquefied, filling the gaps between the weld surfaces of workpieces or between different workpieces. This reinforces weak points in the workpiece structure or securely connects different workpieces. The tightness of the solder filling in the gaps between the weld surfaces directly affects the welding quality. If the solder does not completely fill the gaps, a large amount of air will remain, increasing the surface area exposed to air and making the solder more susceptible to corrosion. Furthermore, the large amount of air remaining in the gaps reduces the density of the solder filling, thus reducing the mechanical strength of the corresponding welded area. Existing visual recognition and inspection of workpiece weld surfaces is limited to identifying weld surfaces that have already been welded; it does not dynamically identify the solder filling status in real time during the welding operation, making it impossible to adjust the welding operation promptly and accurately, and thus failing to ensure the welding quality of the workpiece weld surface. Summary of the Invention

[0003] The purpose of this invention is to provide a machine vision-based weld surface inspection system and method. This system performs multi-target visual recognition on welding operation images of a workpiece to obtain information on the changes in solder morphology on the workpiece weld surface, thereby determining the solder filling state information and comprehensively identifying the filling space and compactness of the solder on the workpiece weld surface. Based on the solder filling state information, it identifies sub-regions on the workpiece weld surface that require secondary welding, and visually recognizes the secondary welding operation images of these sub-regions to obtain information on the flow state of the solder in the sub-regions. It then performs spatial recognition on the solder filling flow during the secondary welding operation to accurately determine the abnormal solder filling locations on the workpiece weld surface. Finally, based on the distribution of abnormal solder filling locations on the workpiece weld surface, it adjusts the welding action of the welding tool on the workpiece weld surface, precisely adjusting the solder flow and filling situation on the workpiece weld surface, effectively filling air gap defects on the workpiece weld surface, and improving the welding quality of the workpiece weld surface.

[0004] This invention is achieved through the following technical solution:

[0005] A machine vision-based weld surface inspection system includes:

[0006] The welding operation multi-target visual recognition module is used to acquire welding operation images of workpieces, perform multi-target visual recognition on the welding operation images, and obtain information on the change of solder morphology on the workpiece welding surface.

[0007] The solder filling state recognition module is used to determine the solder filling state information of the workpiece solder surface based on the solder morphology change information of the workpiece solder surface.

[0008] The secondary welding area determination module is used to determine the sub-area of ​​the workpiece welding surface that needs secondary welding based on the solder filling state information;

[0009] The secondary welding operation visual recognition module is used to collect images of the secondary welding operation of the welding tool on the sub-region, perform visual recognition on the secondary welding operation images, and obtain information on the flow state of the solder in the sub-region.

[0010] The solder filling anomaly location module is used to determine the location of the solder filling anomaly on the workpiece welding surface based on the flow state information.

[0011] The welding action adjustment module is used to adjust the welding action state of the welding tool on the workpiece welding surface based on the abnormal position of the solder filling.

[0012] Optionally, the welding operation multi-target visual recognition module is used to acquire welding operation images of the workpiece, perform multi-target visual recognition on the welding operation images, and obtain information on the changes in the weld morphology of the workpiece weld surface, including:

[0013] A binocular dynamic image of the surface area of ​​the workpiece to be welded is obtained by performing binocular dynamic photography on the surface area; a three-dimensional image of the welding operation is generated based on the binocular parallax of the binocular dynamic image; pixel colorimetric recognition and pixel contour recognition are performed on the three-dimensional image of the welding operation to obtain the pixel colorimetric distribution information and pixel contour distribution information of the three-dimensional image of the welding operation.

[0014] Based on the pixel color distribution information and the pixel contour distribution information, the change information of the contact area between the solder and the workpiece welding surface during the welding operation is obtained, which is used as the solder morphology change information of the workpiece welding surface.

[0015] The solder filling state recognition module is used to determine the solder filling state information of the workpiece solder surface based on the solder morphology change information of the workpiece solder surface, including:

[0016] Based on the change information of the contact area, the boundary information of the solder-filled area of ​​the workpiece welding surface is determined; based on the boundary information of the solder-filled area and the original surface boundary information of the workpiece welding surface, the shape information of the area of ​​the workpiece welding surface filled with solder is determined, which is used as the solder filling state information of the workpiece welding surface.

[0017] Optionally, the secondary welding area determination module is used to determine the sub-area of ​​the workpiece weld surface that requires secondary welding based on the solder filling state information, including:

[0018] The shape information of the area of ​​the workpiece weld surface filled with solder is compared with the shape information of the area of ​​the workpiece weld surface to be welded to determine the gap distribution location information of the workpiece weld surface that is not filled with solder; based on the gap distribution location information, the sub-area of ​​the workpiece weld surface that needs to be welded a second time is determined.

[0019] The secondary welding operation visual recognition module is used to acquire images of the secondary welding operation of the welding tool on the sub-region, perform visual recognition on the secondary welding operation images, and obtain information on the flow state of the solder in the sub-region, including:

[0020] Using the part of the welding tool that contacts the sub-region as a reference, the welding tool is tracked and photographed to obtain a secondary welding operation image of the welding tool in the sub-region; the solder contour is visually recognized from the secondary welding operation image to obtain the flow direction and flow speed information of the solder in the sub-region.

[0021] Optionally, the solder filling anomaly location module is used to determine the location of the solder filling anomaly on the workpiece weld surface based on the flow state information, including:

[0022] Based on the flow direction and velocity information of the solder in the sub-region, the flow trend information of the solder in the sub-region is predicted; wherein, the flow trend information includes the filling volume rate of the gap in the sub-region by the solder in the future time period; the filling volume rate is compared with a preset rate threshold, and if the filling volume rate is less than the preset rate threshold, the location of the corresponding gap is determined as the abnormal solder filling location of the workpiece solder surface.

[0023] The welding action adjustment module is used to adjust the welding action state of the welding tool on the workpiece welding surface based on the abnormal position of the solder filling, including:

[0024] Based on the spatial distribution information of all abnormal solder filling locations on the workpiece weld surface, the movement path of the welding tool for secondary welding of the workpiece weld surface and the relative azimuth angle between the welding tool and the workpiece weld surface are adjusted.

[0025] Optionally, acquiring images of the welding operation on the workpiece also includes:

[0026] Extract the number of abnormal solder filling locations;

[0027] Extract the rate difference between the filling volume rate corresponding to each of the solder filling abnormality locations and the preset rate threshold;

[0028] The data acquisition frequency adjustment coefficient is obtained by using the rate difference between the filling volume rate corresponding to each of the solder filling abnormal locations and the preset rate threshold.

[0029] The data acquisition frequency adjustment coefficient is obtained by the following formula:

[0030]

[0031] Where S represents the data acquisition frequency adjustment coefficient; n represents the number of solder filling abnormal locations; V i V represents the filling volume rate corresponding to the i-th solder filling abnormality location; V0 represents the preset rate threshold; V xi V represents the fill volume rate corresponding to the closest solder fill anomaly location to the i-th solder fill anomaly location; b L represents the standard deviation of the filling volume rate corresponding to n solder filling abnormality locations; b L represents the standard deviation of the linear distance between any two locations out of n solder fill aberration locations; max V represents the maximum distance between any two locations out of n solder fill abnormalities; cmax This represents the maximum rate difference between the filling volume rate corresponding to the abnormal solder filling location and the preset rate threshold; L v The linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold.

[0032] The data acquisition frequency adjustment coefficient is compared with a preset adjustment coefficient threshold.

[0033] When the data acquisition frequency adjustment coefficient exceeds the preset adjustment coefficient threshold, the acquisition frequency of the welding operation image of the workpiece is adjusted to obtain the adjusted acquisition frequency.

[0034] The adjusted sampling frequency is obtained using the following formula:

[0035]

[0036] Where F represents the adjusted acquisition frequency; F0 represents the acquisition frequency before adjustment; S represents the data acquisition frequency adjustment coefficient; S y Indicates the preset adjustment coefficient threshold; L v L represents the linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold, and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold. max V represents the maximum distance between any two locations out of n solder fill abnormalities; max and Vmin These represent the maximum and minimum filling volume rates corresponding to the n solder filling abnormality locations, respectively;

[0037] The welding operation multi-target visual recognition module is controlled to perform image acquisition and operation according to the adjusted acquisition frequency.

[0038] A machine vision-based weld surface inspection method includes:

[0039] Welding operation images of the workpiece are acquired, and multi-target visual recognition is performed on the welding operation images to obtain information on the change in solder morphology on the workpiece welding surface; based on the information on the change in solder morphology on the workpiece welding surface, the solder filling state information of the workpiece welding surface is determined.

[0040] Based on the solder filling state information, the sub-regions of the workpiece weld surface that require secondary welding are determined; images of the secondary welding operation of the welding tool on the sub-regions are acquired, and visual recognition is performed on the secondary welding operation images to obtain the flow state information of the solder in the sub-regions;

[0041] Based on the flow state information, the abnormal location of solder filling on the workpiece welding surface is determined; based on the abnormal location of solder filling, the welding action state of the welding tool on the workpiece welding surface is adjusted.

[0042] Optionally, welding operation images of the workpiece are acquired, and multi-target visual recognition is performed on the welding operation images to obtain solder morphology change information of the workpiece weld surface; based on the solder morphology change information of the workpiece weld surface, the solder filling state information of the workpiece weld surface is determined, including:

[0043] A binocular dynamic image of the surface area of ​​the workpiece to be welded is obtained by performing binocular dynamic photography on the surface area; a three-dimensional image of the welding operation is generated based on the binocular parallax of the binocular dynamic image; pixel colorimetric recognition and pixel contour recognition are performed on the three-dimensional image of the welding operation to obtain the pixel colorimetric distribution information and pixel contour distribution information of the three-dimensional image of the welding operation.

[0044] Based on the pixel color distribution information and the pixel contour distribution information, the change information of the contact area between the solder and the workpiece welding surface during the welding operation is obtained, which is used as the solder morphology change information of the workpiece welding surface.

[0045] Based on the change information of the contact area, the boundary information of the solder-filled area of ​​the workpiece welding surface is determined; based on the boundary information of the solder-filled area and the original surface boundary information of the workpiece welding surface, the shape information of the area of ​​the workpiece welding surface filled with solder is determined, which is used as the solder filling state information of the workpiece welding surface.

[0046] Optionally, based on the solder filling state information, a sub-region of the workpiece weld surface requiring secondary welding is determined; an image of the secondary welding operation of the welding tool on the sub-region is acquired, and visual recognition is performed on the secondary welding operation image to obtain the flow state information of the solder in the sub-region, including:

[0047] The shape information of the area of ​​the workpiece weld surface filled with solder is compared with the shape information of the area of ​​the workpiece weld surface to be welded to determine the gap distribution location information of the workpiece weld surface that is not filled with solder; based on the gap distribution location information, the sub-area of ​​the workpiece weld surface that needs to be welded a second time is determined.

[0048] Using the part of the welding tool that contacts the sub-region as a reference, the welding tool is tracked and photographed to obtain a secondary welding operation image of the welding tool in the sub-region; the solder contour is visually recognized from the secondary welding operation image to obtain the flow direction and flow speed information of the solder in the sub-region.

[0049] Optionally, based on the flow state information, the abnormal location of solder filling on the workpiece weld surface is determined; based on the abnormal solder filling location, the welding action state of the welding tool on the workpiece weld surface is adjusted, including:

[0050] Based on the flow direction and velocity information of the solder in the sub-region, the flow trend information of the solder in the sub-region is predicted; wherein, the flow trend information includes the filling volume rate of the gap in the sub-region by the solder in the future time period; the filling volume rate is compared with a preset rate threshold, and if the filling volume rate is less than the preset rate threshold, the location of the corresponding gap is determined as the abnormal solder filling location of the workpiece solder surface.

[0051] Based on the spatial distribution information of all abnormal solder filling locations on the workpiece weld surface, the movement path of the welding tool for secondary welding of the workpiece weld surface and the relative azimuth angle between the welding tool and the workpiece weld surface are adjusted.

[0052] Optionally, acquiring images of the welding operation on the workpiece also includes:

[0053] Extract the number of abnormal solder filling locations;

[0054] Extract the rate difference between the filling volume rate corresponding to each of the solder filling abnormality locations and the preset rate threshold;

[0055] The data acquisition frequency adjustment coefficient is obtained by using the rate difference between the filling volume rate corresponding to each of the solder filling abnormal locations and the preset rate threshold.

[0056] The data acquisition frequency adjustment coefficient is obtained by the following formula:

[0057]

[0058] Where S represents the data acquisition frequency adjustment coefficient; n represents the number of solder filling abnormal locations; V i V represents the filling volume rate corresponding to the i-th solder filling abnormality location; V0 represents the preset rate threshold; V xi V represents the fill volume rate corresponding to the closest solder fill anomaly location to the i-th solder fill anomaly location; b L represents the standard deviation of the filling volume rate corresponding to n solder filling abnormality locations; b L represents the standard deviation of the linear distance between any two locations out of n solder fill aberration locations; max V represents the maximum distance between any two locations out of n solder fill abnormalities; cmax This represents the maximum rate difference between the filling volume rate corresponding to the abnormal solder filling location and the preset rate threshold; L v The linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold.

[0059] The data acquisition frequency adjustment coefficient is compared with a preset adjustment coefficient threshold.

[0060] When the data acquisition frequency adjustment coefficient exceeds the preset adjustment coefficient threshold, the acquisition frequency of the welding operation image of the workpiece is adjusted to obtain the adjusted acquisition frequency.

[0061] The adjusted sampling frequency is obtained using the following formula:

[0062]

[0063] Where F represents the adjusted acquisition frequency; F0 represents the acquisition frequency before adjustment; S represents the data acquisition frequency adjustment coefficient; S y Indicates the preset adjustment coefficient threshold; L v L represents the linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold, and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold. max V represents the maximum distance between any two locations out of n solder fill abnormalities; max and V minThese represent the maximum and minimum filling volume rates corresponding to the n solder filling abnormality locations, respectively;

[0064] The welding operation multi-target visual recognition module is controlled to perform image acquisition and operation according to the adjusted acquisition frequency.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] This application provides a machine vision-based weld surface inspection system and method that performs multi-target visual recognition on welding operation images of workpieces to obtain information on the changes in solder morphology on the workpiece weld surface, thereby determining the solder filling state information and comprehensively recognizing the filling space and compactness of the solder on the workpiece weld surface. Based on the solder filling state information, the system identifies sub-regions on the workpiece weld surface that require secondary welding, and visually recognizes the secondary welding operation images of these sub-regions to obtain information on the flow state of the solder in the sub-regions. The system performs spatial recognition on the solder filling flow during the secondary welding operation to accurately determine the abnormal solder filling locations on the workpiece weld surface. Based on the distribution of these abnormal solder filling locations, the system adjusts the welding motion of the welding tool on the workpiece weld surface, precisely adjusting the solder flow and filling situation on the workpiece weld surface, effectively filling air gap defects on the workpiece weld surface, and improving the welding quality of the workpiece weld surface. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0068] Figure 1 This is a schematic diagram of a machine vision-based weld surface inspection system provided by the present invention.

[0069] Figure 2 This is a schematic flowchart of a machine vision-based weld surface inspection method provided by the present invention. Detailed Implementation

[0070] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, it should be noted that, for ease of description, only the parts relevant to this application are shown in the accompanying drawings, not the entire structure. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0071] The terms “comprising” and “having”, and any variations thereof, used in this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0072] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0073] Please see Figure 1 As shown in the figure, an embodiment of this application provides a machine vision-based weld surface inspection system. This machine vision-based weld surface inspection system includes:

[0074] The welding operation multi-target visual recognition module is used to acquire welding operation images of workpieces, perform multi-target visual recognition on the welding operation images, and obtain information on the changes in the shape of the solder on the workpiece welding surface.

[0075] The solder filling state recognition module is used to determine the solder filling state information of the workpiece's solder surface based on the solder morphology change information of the workpiece's solder surface.

[0076] The secondary welding area determination module is used to determine the sub-area of ​​the workpiece weld surface that requires secondary welding based on the solder filling status information.

[0077] The secondary welding operation visual recognition module is used to collect images of the secondary welding operation of the welding tool in this sub-region, perform visual recognition on the secondary welding operation images, and obtain information on the flow state of the solder in this sub-region.

[0078] The solder filling anomaly location module is used to determine the location of the solder filling anomaly on the workpiece's weld surface based on the flow state information.

[0079] The welding action adjustment module is used to adjust the welding action state of the welding tool on the workpiece surface based on the abnormal position of the solder filling.

[0080] The beneficial effects of the above embodiments are as follows: the machine vision-based weld surface inspection system performs multi-target visual recognition on the welding operation image of the workpiece to obtain information on the changes in the shape of the solder on the workpiece weld surface, thereby determining the solder filling state information and comprehensively recognizing the filling space and compactness of the solder on the workpiece weld surface; based on the solder filling state information, it determines the sub-regions of the workpiece weld surface that need secondary welding, and visually recognizes the secondary welding operation image of the sub-regions to obtain the flow state information of the solder in the sub-regions, performs spatial recognition on the filling flow of the solder during the secondary welding operation, and accurately determines the abnormal solder filling positions of the workpiece weld surface; then, based on the distribution of abnormal solder filling positions of the workpiece weld surface, it adjusts the welding action state of the welding tool on the workpiece weld surface, precisely adjusts the flow and filling of the solder on the workpiece weld surface, effectively fills the air gap defects of the workpiece weld surface, and improves the welding quality of the workpiece weld surface.

[0081] In another embodiment, the welding operation multi-target visual recognition module is used to acquire welding operation images of the workpiece, perform multi-target visual recognition on the welding operation images, and obtain information on the changes in the weld morphology of the workpiece weld surface, including:

[0082] The surface area of ​​the workpiece to be welded is captured by binocular dynamic imaging to obtain a binocular dynamic image of the surface area; a three-dimensional image of the welding operation is generated based on the binocular parallax of the binocular dynamic image; pixel colorimetric recognition and pixel contour recognition are performed on the three-dimensional image of the welding operation to obtain the pixel colorimetric distribution information and pixel contour distribution information of the three-dimensional image of the welding operation.

[0083] Based on the pixel color distribution information and the pixel contour distribution information, the change information of the contact area between the solder and the workpiece welding surface during the welding operation is obtained, which is used as the solder morphology change information of the workpiece welding surface.

[0084] The solder filling state recognition module is used to determine the solder filling state information of the workpiece's solder surface based on the solder morphology change information, including:

[0085] Based on the change information of the contact area, the boundary information of the solder-filled area of ​​the workpiece welding surface is determined; based on the boundary information of the solder-filled area and the original surface boundary information of the workpiece welding surface, the shape information of the area of ​​the workpiece welding surface filled with solder is determined, which is used as the solder filling state information of the workpiece welding surface.

[0086] The beneficial effects of the above embodiments are as follows: During the welding process on the workpiece surface, the welding tool is first used to heat the solder within the corresponding area of ​​the workpiece surface to be welded, so that the solder, which has become a high-temperature fluid, fills the corresponding gaps or crevices within that area. Since the solder and workpiece have different material types, different colors, and a clear boundary line, the contact area between the solder and workpiece can be accurately determined by identifying the color difference and boundary contour between them. Therefore, binocular dynamic imaging is performed on the surface area of ​​the workpiece to be welded, resulting in a binocular dynamic image of the surface area, and a three-dimensional image of the welding operation is generated. Then, pixel colorimetric recognition and pixel contour recognition are performed on the three-dimensional image of the welding operation to obtain detailed pixel colorimetric distribution and pixel contour distribution information, thereby distinguishing the workpiece portion and the solder portion of the surface area to be welded. Based on the pixel color distribution information and pixel contour distribution information, the change in the contact area between the solder and the workpiece weld surface during the welding operation is obtained, i.e., the change in the area of ​​the contact area between the solder and the workpiece weld surface. Spatial calibration is then performed on the contact state between the solder and the workpiece weld surface during the first welding operation. Furthermore, based on this contact area change information, the boundary information of the solder-filled area on the workpiece weld surface is determined. Based on this solder-filled area boundary information and the original surface boundary information of the workpiece weld surface, the shape information of the area filled by the solder on the workpiece weld surface is determined, thereby accurately identifying the solder-filled space on the workpiece weld surface.

[0087] In another embodiment, the secondary welding area determination module is used to determine the sub-area of ​​the workpiece weld surface that requires secondary welding based on the solder filling state information, including:

[0088] The shape information of the area of ​​the workpiece weld surface filled with solder is compared with the shape information of the area of ​​the workpiece weld surface to be welded to determine the location information of the gaps on the workpiece weld surface that are not filled with solder; based on the location information of the gaps, the sub-areas of the workpiece weld surface that need to be welded a second time are determined.

[0089] The secondary welding operation visual recognition module is used to acquire images of the secondary welding operation of the welding tool in this sub-region, and to perform visual recognition on these images to obtain information on the flow state of the solder in this sub-region, including:

[0090] Using the part of the welding tool that contacts the sub-region as a reference, the welding tool is tracked and photographed to obtain an image of the welding tool's secondary welding operation in the sub-region; the weld contour is visually recognized from the secondary welding operation image to obtain information on the flow direction and flow speed of the weld in the sub-region.

[0091] The beneficial effects of the above embodiments are as follows: During the initial welding operation on the workpiece surface, factors such as welding operation precision and low solder heating temperature prevent the welding area of ​​the workpiece surface from being completely and tightly filled with solder, resulting in air gaps (i.e., gaps filled by solder) in the welding area. Based on the distribution information of all unfilled gaps on the workpiece surface, the area of ​​the workpiece surface that needs to be re-welded is calibrated, and the sub-area that needs secondary welding is determined, defining the area range for subsequent secondary welding operations. Furthermore, using the part of the welding tool that contacts this sub-area as a reference, the welding tool is tracked and photographed to obtain an image of the secondary welding operation of the welding tool on this sub-area, comprehensively and accurately recording the changes in solder morphology during the secondary welding operation. Solder contour visual recognition is then performed on this secondary welding operation image to obtain the flow direction and velocity information of the solder within the sub-area, enabling spatial identification of the solder filling flow during the secondary welding operation.

[0092] In another embodiment, the solder filling anomaly location module is used to determine the location of the solder filling anomaly on the workpiece solder surface based on the flow state information, including:

[0093] Based on the flow direction and flow velocity information of the solder in the sub-region, the flow trend information of the solder in the sub-region is predicted; wherein, the flow trend information includes the filling volume rate of the gap in the sub-region by the solder in the future time period; the filling volume rate is compared with a preset rate threshold, and if the filling volume rate is less than the preset rate threshold, the location of the corresponding gap is determined as the abnormal solder filling location of the workpiece welding surface.

[0094] The welding action adjustment module is used to adjust the welding action state of the welding tool on the workpiece surface based on the abnormal location of the solder filling, including:

[0095] Based on the spatial distribution information of all abnormal solder filling locations on the workpiece weld surface, adjust the movement path of the welding tool for secondary welding on the workpiece weld surface and the relative azimuth angle between the welding tool and the workpiece weld surface.

[0096] The beneficial effects of the above embodiments are that the flow direction and flow speed information of the solder in the sub-region of the workpiece surface that needs to be welded a second time reflect the filling dynamic characteristics of the solder during the second welding process. In this way, the filling volume rate of the solder in the sub-region in the future time period (i.e., the volume of solder flowing to fill the gap in the sub-region per unit time) is predicted. The filling volume rate is compared with a threshold to identify and determine the abnormal solder filling position of the workpiece surface, providing a reliable and accurate basis for subsequent real-time adjustment of the second welding. Furthermore, based on the spatial distribution information of all solder filling anomalies on the workpiece weld surface, the movement path of the welding tool for secondary welding of the workpiece weld surface is adjusted. This allows the welding tool to heat the solder at the solder filling anomalies on the workpiece weld surface during the secondary welding process, ensuring that the solder can accurately fill the air gaps in the sub-region and improve the tightness of the solder filling on the workpiece weld surface. Moreover, based on the spatial distribution information of all solder filling anomalies on the workpiece weld surface, the relative azimuth angle between the welding tool and the workpiece weld surface is adjusted, allowing the welding tool to accurately guide the solder to the air gaps in the sub-region, effectively filling the air gap defects on the workpiece weld surface and improving the welding quality of the workpiece weld surface.

[0097] In another embodiment, acquiring images of the welding operation on the workpiece further includes:

[0098] Extract the number of abnormal solder filling locations;

[0099] Extract the rate difference between the filling volume rate corresponding to each of the solder filling abnormality locations and the preset rate threshold;

[0100] The data acquisition frequency adjustment coefficient is obtained by using the rate difference between the filling volume rate corresponding to each of the solder filling abnormal locations and the preset rate threshold.

[0101] The data acquisition frequency adjustment coefficient is obtained by the following formula:

[0102]

[0103] Where S represents the data acquisition frequency adjustment coefficient; n represents the number of solder filling abnormal locations; V i V represents the filling volume rate corresponding to the i-th solder filling abnormality location; V0 represents the preset rate threshold; V xi V represents the fill volume rate corresponding to the closest solder fill anomaly location to the i-th solder fill anomaly location; b L represents the standard deviation of the filling volume rate corresponding to n solder filling abnormality locations; b L represents the standard deviation of the linear distance between any two locations out of n solder fill aberration locations; maxV represents the maximum distance between any two locations out of n solder fill abnormalities; cmax This represents the maximum rate difference between the filling volume rate corresponding to the abnormal solder filling location and the preset rate threshold; L v The linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold.

[0104] The data acquisition frequency adjustment coefficient is compared with a preset adjustment coefficient threshold.

[0105] When the data acquisition frequency adjustment coefficient exceeds the preset adjustment coefficient threshold, the acquisition frequency of the welding operation image of the workpiece is adjusted to obtain the adjusted acquisition frequency.

[0106] The adjusted sampling frequency is obtained using the following formula:

[0107]

[0108] Where F represents the adjusted acquisition frequency; F0 represents the acquisition frequency before adjustment; S represents the data acquisition frequency adjustment coefficient; S y Indicates the preset adjustment coefficient threshold; L v L represents the linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold, and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold. max V represents the maximum distance between any two locations out of n solder fill abnormalities; max and V min These represent the maximum and minimum filling volume rates corresponding to the n solder filling abnormality locations, respectively;

[0109] The welding operation multi-target visual recognition module is controlled to perform image acquisition and operation according to the adjusted acquisition frequency.

[0110] The beneficial effects of the above embodiments are that, by extracting the number of solder filling anomalies and the rate difference between their corresponding filling volume rates and preset rate thresholds, this technical solution can dynamically evaluate anomalies in the welding process. Based on these evaluation results, the acquisition frequency is adjusted using a data acquisition frequency adjustment coefficient, thereby achieving efficient and targeted acquisition of welding operation images. The adjustment of the acquisition frequency is based not only on the number of solder filling anomalies but also on multiple factors such as differences in filling volume rates, distance distribution between anomaly locations, and the distance between the maximum and minimum rate difference locations. This multi-dimensional consideration makes the adjusted acquisition frequency more accurately reflect the actual situation in the welding process, improving the accuracy and efficiency of monitoring. Since the data acquisition frequency adjustment coefficient is dynamically calculated based on real-time data during the welding process, this technical solution has strong adaptability. It can automatically adjust the acquisition frequency according to different welding conditions and anomalies, thereby adapting to various complex welding environments. By adjusting the acquisition frequency, this technical solution can avoid excessive data acquisition at unnecessary times, thereby saving storage space and processing resources. Meanwhile, because the acquisition frequency is adjusted according to actual conditions, it ensures that sufficient information can be obtained at critical moments, providing strong support for subsequent fault analysis and quality control. Through precise acquisition and analysis of welding operation images, this technical solution can promptly identify potential problems in the welding process, such as insufficient solder filling, excessively fast or slow welding speeds. If these problems are corrected in a timely manner, it will help improve welding quality and reduce the generation of defective products.

[0111] In summary, the technical benefits of this solution in terms of performance indicators are mainly reflected in dynamically adjusting the data acquisition frequency, improving monitoring accuracy and efficiency, enhancing adaptability, optimizing resource utilization, and improving welding quality. These effects work together to monitor and control the welding process, providing strong technical support for improving welding quality.

[0112] Please see Figure 2 As shown, an embodiment of this application provides a machine vision-based weld surface inspection method. This machine vision-based weld surface inspection method includes:

[0113] The welding operation image of the workpiece is acquired, and multi-target visual recognition is performed on the welding operation image to obtain the information on the change of solder morphology on the workpiece welding surface; based on the information on the change of solder morphology on the workpiece welding surface, the information on the solder filling state of the workpiece welding surface is determined.

[0114] Based on the solder filling state information, the sub-area of ​​the workpiece surface that needs secondary welding is determined; the secondary welding operation image of the welding tool in the sub-area is acquired, and the secondary welding operation image is visually recognized to obtain the flow state information of the solder in the sub-area;

[0115] Based on the flow state information, the abnormal location of solder filling on the workpiece's weld surface is determined; based on the abnormal location of solder filling, the welding action state of the welding tool on the workpiece's weld surface is adjusted.

[0116] The beneficial effects of the above embodiments are as follows: This machine vision-based weld surface detection method performs multi-target visual recognition on the welding operation image of the workpiece to obtain information on the changes in the shape of the solder on the workpiece weld surface, thereby determining the solder filling state information and comprehensively recognizing the filling space and compactness of the solder on the workpiece weld surface; based on the solder filling state information, it determines the sub-regions of the workpiece weld surface that require secondary welding, and visually recognizes the secondary welding operation image of the sub-regions to obtain the flow state information of the solder in the sub-regions; it performs spatial recognition on the filling flow of the solder during the secondary welding operation to accurately determine the abnormal solder filling positions on the workpiece weld surface; and based on the distribution of abnormal solder filling positions on the workpiece weld surface, it adjusts the welding action state of the welding tool on the workpiece weld surface, precisely adjusts the flow and filling of the solder on the workpiece weld surface, effectively fills the air gap defects on the workpiece weld surface, and improves the welding quality of the workpiece weld surface.

[0117] In another embodiment, welding operation images of the workpiece are acquired, and multi-target visual recognition is performed on the welding operation images to obtain solder morphology change information of the workpiece welding surface; based on the solder morphology change information of the workpiece welding surface, solder filling state information of the workpiece welding surface is determined, including:

[0118] The surface area of ​​the workpiece to be welded is captured by binocular dynamic imaging to obtain a binocular dynamic image of the surface area; a three-dimensional image of the welding operation is generated based on the binocular parallax of the binocular dynamic image; pixel colorimetric recognition and pixel contour recognition are performed on the three-dimensional image of the welding operation to obtain the pixel colorimetric distribution information and pixel contour distribution information of the three-dimensional image of the welding operation.

[0119] Based on the pixel color distribution information and the pixel contour distribution information, the change information of the contact area between the solder and the workpiece welding surface during the welding operation is obtained, which is used as the solder morphology change information of the workpiece welding surface.

[0120] Based on the change information of the contact area, the boundary information of the solder-filled area of ​​the workpiece welding surface is determined; based on the boundary information of the solder-filled area and the original surface boundary information of the workpiece welding surface, the shape information of the area of ​​the workpiece welding surface filled with solder is determined, which is used as the solder filling state information of the workpiece welding surface.

[0121] The beneficial effects of the above embodiments are as follows: During the welding process on the workpiece surface, the welding tool is first used to heat the solder within the corresponding area of ​​the workpiece surface to be welded, so that the solder, which has become a high-temperature fluid, fills the corresponding gaps or crevices within that area. Since the solder and workpiece have different material types, different colors, and a clear boundary line, the contact area between the solder and workpiece can be accurately determined by identifying the color difference and boundary contour between them. Therefore, binocular dynamic imaging is performed on the surface area of ​​the workpiece to be welded, resulting in a binocular dynamic image of the surface area, and a three-dimensional image of the welding operation is generated. Then, pixel colorimetric recognition and pixel contour recognition are performed on the three-dimensional image of the welding operation to obtain detailed pixel colorimetric distribution and pixel contour distribution information, thereby distinguishing the workpiece portion and the solder portion of the surface area to be welded. Based on the pixel color distribution information and pixel contour distribution information, the change in the contact area between the solder and the workpiece weld surface during the welding operation is obtained, i.e., the change in the area of ​​the contact area between the solder and the workpiece weld surface. Spatial calibration is then performed on the contact state between the solder and the workpiece weld surface during the first welding operation. Furthermore, based on this contact area change information, the boundary information of the solder-filled area on the workpiece weld surface is determined. Based on this solder-filled area boundary information and the original surface boundary information of the workpiece weld surface, the shape information of the area filled by the solder on the workpiece weld surface is determined, thereby accurately identifying the solder-filled space on the workpiece weld surface.

[0122] In another embodiment, based on the solder filling state information, the sub-region of the workpiece weld surface that requires secondary welding is determined; an image of the secondary welding operation of the welding tool in the sub-region is acquired, and visual recognition is performed on the secondary welding operation image to obtain the flow state information of the solder in the sub-region, including:

[0123] The shape information of the area of ​​the workpiece weld surface filled with solder is compared with the shape information of the area of ​​the workpiece weld surface to be welded to determine the location information of the gaps on the workpiece weld surface that are not filled with solder; based on the location information of the gaps, the sub-areas of the workpiece weld surface that need to be welded a second time are determined.

[0124] Using the part of the welding tool that contacts the sub-region as a reference, the welding tool is tracked and photographed to obtain an image of the welding tool's secondary welding operation in the sub-region; the weld contour is visually recognized from the secondary welding operation image to obtain information on the flow direction and flow speed of the weld in the sub-region.

[0125] The beneficial effects of the above embodiments are as follows: During the initial welding operation on the workpiece surface, factors such as welding operation precision and low solder heating temperature prevent the welding area of ​​the workpiece surface from being completely and tightly filled with solder, resulting in air gaps (i.e., gaps filled by solder) in the welding area. Based on the distribution information of all unfilled gaps on the workpiece surface, the area of ​​the workpiece surface that needs to be re-welded is calibrated, and the sub-area that needs secondary welding is determined, defining the area range for subsequent secondary welding operations. Furthermore, using the part of the welding tool that contacts this sub-area as a reference, the welding tool is tracked and photographed to obtain an image of the secondary welding operation of the welding tool on this sub-area, comprehensively and accurately recording the changes in solder morphology during the secondary welding operation. Solder contour visual recognition is then performed on this secondary welding operation image to obtain the flow direction and velocity information of the solder within the sub-area, enabling spatial identification of the solder filling flow during the secondary welding operation.

[0126] In another embodiment, based on the flow state information, the location of abnormal solder filling on the workpiece weld surface is determined; based on the abnormal solder filling location, the welding action state of the welding tool on the workpiece weld surface is adjusted, including:

[0127] Based on the flow direction and flow velocity information of the solder in the sub-region, the flow trend information of the solder in the sub-region is predicted; wherein, the flow trend information includes the filling volume rate of the gap in the sub-region by the solder in the future time period; the filling volume rate is compared with a preset rate threshold, and if the filling volume rate is less than the preset rate threshold, the location of the corresponding gap is determined as the abnormal solder filling location of the workpiece welding surface.

[0128] Based on the spatial distribution information of all abnormal solder filling locations on the workpiece weld surface, adjust the movement path of the welding tool for secondary welding on the workpiece weld surface and the relative azimuth angle between the welding tool and the workpiece weld surface.

[0129] The beneficial effects of the above embodiments are that the flow direction and flow speed information of the solder in the sub-region of the workpiece surface that needs to be welded a second time reflect the filling dynamic characteristics of the solder during the second welding process. In this way, the filling volume rate of the solder in the sub-region in the future time period (i.e., the volume of solder flowing to fill the gap in the sub-region per unit time) is predicted. The filling volume rate is compared with a threshold to identify and determine the abnormal solder filling position of the workpiece surface, providing a reliable and accurate basis for subsequent real-time adjustment of the second welding. Furthermore, based on the spatial distribution information of all solder filling anomalies on the workpiece weld surface, the movement path of the welding tool for secondary welding of the workpiece weld surface is adjusted. This allows the welding tool to heat the solder at the solder filling anomalies on the workpiece weld surface during the secondary welding process, ensuring that the solder can accurately fill the air gaps in the sub-region and improve the tightness of the solder filling on the workpiece weld surface. Moreover, based on the spatial distribution information of all solder filling anomalies on the workpiece weld surface, the relative azimuth angle between the welding tool and the workpiece weld surface is adjusted, allowing the welding tool to accurately guide the solder to the air gaps in the sub-region, effectively filling the air gap defects on the workpiece weld surface and improving the welding quality of the workpiece weld surface.

[0130] In another embodiment, acquiring images of the welding operation on the workpiece further includes:

[0131] Extract the number of abnormal solder filling locations;

[0132] Extract the rate difference between the filling volume rate corresponding to each of the solder filling abnormality locations and the preset rate threshold;

[0133] The data acquisition frequency adjustment coefficient is obtained by using the rate difference between the filling volume rate corresponding to each of the solder filling abnormal locations and the preset rate threshold.

[0134] The data acquisition frequency adjustment coefficient is obtained by the following formula:

[0135]

[0136] Where S represents the data acquisition frequency adjustment coefficient; n represents the number of solder filling abnormal locations; V i V represents the filling volume rate corresponding to the i-th solder filling abnormality location; V0 represents the preset rate threshold; V xi V represents the fill volume rate corresponding to the closest solder fill anomaly location to the i-th solder fill anomaly location; b L represents the standard deviation of the filling volume rate corresponding to n solder filling abnormality locations; b L represents the standard deviation of the linear distance between any two locations out of n solder fill aberration locations; maxV represents the maximum distance between any two locations out of n solder fill abnormalities; cmax This represents the maximum rate difference between the filling volume rate corresponding to the abnormal solder filling location and the preset rate threshold; L v The linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold.

[0137] The data acquisition frequency adjustment coefficient is compared with a preset adjustment coefficient threshold.

[0138] When the data acquisition frequency adjustment coefficient exceeds the preset adjustment coefficient threshold, the acquisition frequency of the welding operation image of the workpiece is adjusted to obtain the adjusted acquisition frequency.

[0139] The adjusted sampling frequency is obtained using the following formula:

[0140]

[0141] Where F represents the adjusted acquisition frequency; F0 represents the acquisition frequency before adjustment; S represents the data acquisition frequency adjustment coefficient; S y Indicates the preset adjustment coefficient threshold; L v L represents the linear distance between the solder filling anomaly location corresponding to the maximum rate difference between the filling volume rate and the preset rate threshold, and the solder filling anomaly location corresponding to the minimum rate difference between the filling volume rate and the preset rate threshold. max V represents the maximum distance between any two locations out of n solder fill abnormalities; max and V min These represent the maximum and minimum filling volume rates corresponding to the n solder filling abnormality locations, respectively;

[0142] The welding operation multi-target visual recognition module is controlled to perform image acquisition and operation according to the adjusted acquisition frequency.

[0143] The beneficial effects of the above embodiments are that, by extracting the number of solder filling anomalies and the rate difference between their corresponding filling volume rates and preset rate thresholds, this technical solution can dynamically evaluate anomalies in the welding process. Based on these evaluation results, the acquisition frequency is adjusted using a data acquisition frequency adjustment coefficient, thereby achieving efficient and targeted acquisition of welding operation images. The adjustment of the acquisition frequency is based not only on the number of solder filling anomalies but also on multiple factors such as differences in filling volume rates, distance distribution between anomaly locations, and the distance between the maximum and minimum rate difference locations. This multi-dimensional consideration makes the adjusted acquisition frequency more accurately reflect the actual situation in the welding process, improving the accuracy and efficiency of monitoring. Since the data acquisition frequency adjustment coefficient is dynamically calculated based on real-time data during the welding process, this technical solution has strong adaptability. It can automatically adjust the acquisition frequency according to different welding conditions and anomalies, thereby adapting to various complex welding environments. By adjusting the acquisition frequency, this technical solution can avoid excessive data acquisition at unnecessary times, thereby saving storage space and processing resources. Meanwhile, because the acquisition frequency is adjusted according to actual conditions, it ensures that sufficient information can be obtained at critical moments, providing strong support for subsequent fault analysis and quality control. Through precise acquisition and analysis of welding operation images, this technical solution can promptly identify potential problems in the welding process, such as insufficient solder filling, excessively fast or slow welding speeds. If these problems are corrected in a timely manner, it will help improve welding quality and reduce the generation of defective products.

[0144] In summary, the technical benefits of this solution in terms of performance indicators are mainly reflected in dynamically adjusting the data acquisition frequency, improving monitoring accuracy and efficiency, enhancing adaptability, optimizing resource utilization, and improving welding quality. These effects work together to monitor and control the welding process, providing strong technical support for improving welding quality.

[0145] In summary, this machine vision-based weld surface inspection system and method performs multi-target visual recognition on the welding operation images of the workpiece to obtain information on the changes in the solder morphology of the workpiece weld surface, thereby determining the solder filling state information and comprehensively recognizing the filling space and compactness of the solder on the workpiece weld surface. Based on the solder filling state information, it identifies the sub-regions of the workpiece weld surface that require secondary welding, and visually recognizes the secondary welding operation images of the sub-regions to obtain the flow state information of the solder in the sub-regions. It performs spatial recognition on the filling flow of the solder during the secondary welding operation to accurately determine the abnormal solder filling locations on the workpiece weld surface. Then, based on the distribution of abnormal solder filling locations on the workpiece weld surface, it adjusts the welding action state of the welding tool on the workpiece weld surface, precisely adjusts the flow and filling of the solder on the workpiece weld surface, effectively fills the air gap defects on the workpiece weld surface, and improves the welding quality of the workpiece weld surface.

[0146] The above is only one specific embodiment of the present invention, and any improvements made based on the concept of the present invention shall be considered within the scope of protection of the present invention.

Claims

1. A machine vision-based weld face inspection system, characterized by, The welding operation multi-target visual recognition module is used for collecting welding operation images of a workpiece, performing multi-target visual recognition on the welding operation images, and obtaining solder morphology change information of a workpiece welding surface. The solder filling state recognition module is used for determining solder filling state information of the workpiece welding surface based on the solder morphology change information of the workpiece welding surface. The secondary welding area determination module is used for determining a sub-area of the workpiece welding surface that needs secondary welding based on the solder filling state information. The secondary welding operation visual recognition module is used for collecting secondary welding operation images of the sub-area by a welding tool, performing visual recognition on the secondary welding operation images, and obtaining flow state information of the solder in the sub-area. The solder filling abnormality positioning module is used for determining a solder filling abnormality position of the workpiece welding surface based on the flow state information. The welding action adjustment module is used for adjusting a welding action state of the welding tool on the workpiece welding surface based on the solder filling abnormality position. The welding operation multi-target visual recognition module is used for collecting welding operation images of a workpiece, performing multi-target visual recognition on the welding operation images, and obtaining solder morphology change information of a workpiece welding surface, including: The surface area of the workpiece subjected to the welding operation is binocularly dynamically photographed to obtain binocular dynamic images of the surface area; a welding operation three-dimensional image is generated based on binocular parallax of the binocular dynamic images; pixel chrominance recognition and pixel contour recognition are performed on the welding operation three-dimensional image to obtain pixel chrominance distribution information and pixel contour distribution information of the welding operation three-dimensional image; Based on the pixel chrominance distribution information and the pixel contour distribution information, contact area range change information of the solder and the workpiece welding surface during the welding operation process is obtained as solder morphology change information of the workpiece welding surface; The solder filling state recognition module is used for determining solder filling state information of the workpiece welding surface based on the solder morphology change information of the workpiece welding surface, including: Based on the contact area range change information, solder filled area boundary information of the workpiece welding surface is determined; based on the solder filled area boundary information and original surface boundary information of the workpiece welding surface, region shape information of the workpiece welding surface filled with the solder is determined as the solder filling state information of the workpiece welding surface; The solder filling abnormality positioning module is used for determining a solder filling abnormality position of the workpiece welding surface based on the flow state information, including: Based on flow direction and flow speed information of the solder in the sub-area, flow trend information of the solder in the sub-area is predicted; wherein the flow trend information includes a filling volume rate of the solder to gaps in the sub-area in a future time period; the filling volume rate is compared with a preset rate threshold value, and if the filling volume rate is less than the preset rate threshold value, a position corresponding to the gap is determined as the solder filling abnormality position of the workpiece welding surface; The welding action adjustment module is used for adjusting a welding action state of the welding tool on the workpiece welding surface based on the solder filling abnormality position, including: ​ Adjust a moving path state of the welding tool for performing secondary welding on the workpiece welding surface and a relative angle state of the welding tool and the workpiece welding surface based on spatial distribution information of all abnormal solder filling positions on the workpiece welding surface. 2.The machine vision-based welding surface detection system according to claim 1, wherein: The secondary welding area determination module is configured to determine a sub-area of the workpiece welding surface that needs secondary welding based on the solder filling state information, including: Comparing the area shape information of the workpiece welding surface filled with solder with the shape information of the welding area of the workpiece, determine the gap distribution position information of the workpiece welding surface that is not filled with solder; based on the gap distribution position information, determine the sub-area of the workpiece welding surface that needs secondary welding; The secondary welding operation visual recognition module is configured to collect secondary welding operation images of the welding tool on the sub-area, and perform visual recognition on the secondary welding operation images to obtain solder flow state information in the sub-area, including: Taking the part of the welding tool that contacts the sub-area as a reference, track and capture the welding tool to obtain secondary welding operation images of the welding tool on the sub-area; perform solder profile visual recognition on the secondary welding operation images to obtain the flow direction and flow speed information of the solder in the sub-area. 3.The machine vision-based welding surface detection system according to claim 1, wherein: Collecting welding operation images of the workpiece further includes: Extracting the number of abnormal solder filling positions; Extracting the rate difference between the filling volume rate corresponding to each of the abnormal solder filling positions and the preset rate threshold; Using the rate difference between the filling volume rate corresponding to each of the abnormal solder filling positions and the preset rate threshold to obtain a data collection frequency adjustment coefficient; Wherein, the data collection frequency adjustment coefficient is obtained by the following formula: ， wherein S represents a data collection frequency adjustment coefficient; n represents a number of solder filling abnormal positions; V i represents a filling volume rate corresponding to the i-th solder filling abnormal position; V0 represents a preset rate threshold; V xi represents a filling volume rate corresponding to another solder filling abnormal position closest to the i-th solder filling abnormal position; V b represents a standard deviation of filling volume rates corresponding to n solder filling abnormal positions; L b represents a standard deviation of linear interval distances between each two positions in the n solder filling abnormal positions; L max represents a maximum value of interval distances between each two positions in the n solder filling abnormal positions; V cmax represents a maximum value of rate difference values between the filling volume rate corresponding to the solder filling abnormal position and the preset rate threshold; L v represents a linear interval distance between a solder filling abnormal position corresponding to a maximum value of rate difference values between the filling volume rate and the preset rate threshold and a solder filling abnormal position corresponding to a minimum value of rate difference values between the filling volume rate and the preset rate threshold. Compare the data collection frequency adjustment coefficient with a preset adjustment coefficient threshold; When the data collection frequency adjustment coefficient exceeds the preset adjustment coefficient threshold, adjust the collection frequency of the welding operation images of the workpiece to obtain an adjusted collection frequency; Wherein, the adjusted collection frequency is obtained by the following formula: ， Wherein, F represents the adjusted collection frequency; F0 represents the collection frequency before adjustment; S represents the data collection frequency adjustment coefficient; S y represents the preset adjustment coefficient threshold; L v represents the linear interval distance between the solder filling abnormal position corresponding to the maximum rate difference value between the filling volume rate and the preset rate threshold and the solder filling abnormal position corresponding to the minimum rate difference value between the filling volume rate and the preset rate threshold; L max represents the maximum value of the interval distance between each two positions in the n solder filling abnormal positions; V max and V min respectively represent the maximum and minimum values of the filling volume rate corresponding to the n solder filling abnormal positions; Control the welding operation multi-target visual recognition module to collect images according to the adjusted collection frequency.

4. A method of weld face inspection based on machine vision, characterized by, Including: Collecting welding operation images of the workpiece, and performing multi-target visual recognition on the welding operation images to obtain solder morphology change information of the workpiece welding surface; Determine solder filling state information of the workpiece welding surface based on the solder morphology change information of the workpiece welding surface; Determine a sub-area of the workpiece welding surface that needs secondary welding based on the solder filling state information; collect secondary welding operation images of the welding tool on the sub-area, and perform visual recognition on the secondary welding operation images to obtain solder flow state information in the sub-area; Determine abnormal solder filling positions of the workpiece welding surface based on the flow state information; and adjust the welding action state of the welding tool on the workpiece welding surface based on the abnormal solder filling positions. Collecting welding operation images of the workpiece, performing multi-target visual recognition on the welding operation images to obtain solder form change information of the workpiece welding surface; Based on the solder form change information of the workpiece welding surface, determining the solder filling state information of the workpiece welding surface, including: Double dynamic shooting of the surface area of the workpiece subjected to welding operation to obtain double dynamic images of the surface area; generating a three-dimensional image of the welding operation based on the binocular disparity of the double dynamic images; performing pixel chroma recognition and pixel contour recognition on the three-dimensional image of the welding operation to obtain pixel chroma distribution information and pixel contour distribution information of the three-dimensional image of the welding operation; Based on the pixel chroma distribution information and the pixel contour distribution information, obtaining the contact area range change information of the solder and the workpiece welding surface during the welding operation process, which is used as the solder form change information of the workpiece welding surface; Based on the contact area range change information, determining the solder filled area boundary information of the workpiece welding surface; based on the solder filled area boundary information and the original surface boundary information of the workpiece welding surface, determining the area shape information of the workpiece welding surface filled with solder, which is used as the solder filling state information of the workpiece welding surface; Based on the flow state information, determining the solder filling abnormal position of the workpiece welding surface; based on the solder filling abnormal position, adjusting the welding action state of the welding tool to the workpiece welding surface, including: Based on the flow direction and flow speed information of the solder in the sub-area, predicting the flow trend information of the solder in the sub-area; wherein the flow trend information includes the filling volume rate of the solder to the gap in the sub-area in the future time period; comparing the filling volume rate with the preset rate threshold value, if the filling volume rate is less than the preset rate threshold value, the position of the corresponding gap is determined as the solder filling abnormal position of the workpiece welding surface; Based on the spatial distribution information of all solder filling abnormal positions on the workpiece welding surface, adjusting the movement path state of the welding tool for secondary welding of the workpiece welding surface and the relative angle state of the welding tool and the workpiece welding surface.

5. The machine vision-based welding surface detection method of claim 4, wherein: Based on the solder filling state information, determining the sub-area of the workpiece welding surface that needs secondary welding; collecting secondary welding operation images of the welding tool to the sub-area, performing visual recognition on the secondary welding operation images to obtain the flow state information of the solder in the sub-area, including: Comparing the area shape information of the workpiece welding surface filled with solder with the welding area shape information of the workpiece welding surface to determine the gap distribution position information of the workpiece welding surface not filled with solder; based on the gap distribution position information, determining the sub-area of the workpiece welding surface that needs secondary welding; Taking the part of the welding tool contacting the sub-area as a reference, tracking and shooting the welding tool to obtain the secondary welding operation images of the welding tool to the sub-area; performing solder contour visual recognition on the secondary welding operation images to obtain the flow direction and flow speed information of the solder in the sub-area. 6.The method of claim 4, wherein the collecting the welding operation image of the workpiece further comprises: extracting a number of abnormal solder filling positions; extracting a rate difference between a filling volume rate corresponding to each of the abnormal solder filling positions and the preset rate threshold value; acquiring a data collection frequency adjustment coefficient using the rate difference between the filling volume rate corresponding to each of the abnormal solder filling positions and the preset rate threshold value; wherein the data collection frequency adjustment coefficient is acquired by the following formula: comparing the data collection frequency adjustment coefficient with a preset adjustment coefficient threshold value; ， wherein S represents a data collection frequency adjustment coefficient; n represents a number of solder filling abnormal positions; V i represents a filling volume rate corresponding to the i-th solder filling abnormal position; V0 represents a preset rate threshold; V xi represents a filling volume rate corresponding to another solder filling abnormal position closest to the i-th solder filling abnormal position; V b represents a standard deviation of filling volume rates corresponding to n solder filling abnormal positions; L b represents a standard deviation of linear interval distances between each two positions in n solder filling abnormal positions; L max represents a maximum value of interval distances between each two positions in n solder filling abnormal positions; V cmax represents a maximum value of rate difference values between the filling volume rate corresponding to the solder filling abnormal position and the preset rate threshold; L v represents a linear interval distance between a solder filling abnormal position corresponding to a maximum value of rate difference values between the filling volume rate and the preset rate threshold and a solder filling abnormal position corresponding to a minimum value of rate difference values between the filling volume rate and the preset rate threshold. when the data collection frequency adjustment coefficient exceeds the preset adjustment coefficient threshold value, adjusting a collection frequency of the welding operation image of the workpiece to obtain an adjusted collection frequency; wherein the adjusted collection frequency is acquired by the following formula: controlling the welding operation multi-target visual recognition module to perform image collection operation according to the adjusted collection frequency. ， Wherein, F represents the adjusted collection frequency; F0 represents the collection frequency before adjustment; S represents the data collection frequency adjustment coefficient; S y represents the preset adjustment coefficient threshold; L v represents the linear interval distance between the solder filling abnormal position corresponding to the maximum rate difference value between the filling volume rate and the preset rate threshold and the solder filling abnormal position corresponding to the minimum rate difference value between the filling volume rate and the preset rate threshold; L max represents the maximum value of the interval distance between each two positions in the n solder filling abnormal positions; V max and V min respectively represent the maximum and minimum values of the filling volume rate corresponding to the n solder filling abnormal positions; ​

Images