Vision detection method, device and readable storage medium based on laser welding

Abstract

The present application relates to the technical field of laser welding, and provides a visual detection method and device based on laser welding and a readable storage medium, the method comprising: controlling a galvanometer welding head to weld according to a preset standard welding image to obtain a plurality of welding image samples; comparing and analyzing each welding image sample with the standard welding image to obtain welding data of the welding image sample, and labeling unqualified welding sections in the welding data to obtain labeled welding data; obtaining welding process parameters of the unqualified welding sections in the welding image sample, and associating the welding process parameters with the welding data of the welding image sample to obtain training data; constructing and training a visual detection model; inputting a collected welding image to be detected into the trained visual detection model, and the visual detection model judging whether the welding image to be detected has unqualified welding sections. The technical scheme provided in the present application embodiment can effectively improve the detection efficiency of visual detection welding effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of laser welding, and in particular to a visual detection method and device based on laser welding and a readable storage medium. BACKGROUND

[0002] Laser galvanometer welding is a relatively leading welding technology in the current welding process. The welding pattern formed by laser galvanometer welding needs to be detected to determine whether the welding process is qualified.

[0003] At present, the welding pattern is affected by factors such as shaft motion error, laser energy change, layer overlap, and material unevenness, and there is a certain deviation between the welding pattern and the expected pattern, especially at the interface of the welding pattern. At present, the industry mainly relies on human eyes to observe under a microscope to judge the quality of the interface. In actual application scenarios, the inventors have found that the detection efficiency of the welding effect is low due to the limitation of the quality of the detection personnel. SUMMARY

[0004] Therefore, the embodiments of the present application provide a visual detection method and device based on laser welding and a readable storage medium, which can effectively improve the detection efficiency of visual detection of the welding effect.

[0005] To achieve the above-mentioned purpose, in a first aspect, a visual detection method based on laser welding is provided, and the method comprises:

[0006] controlling a galvanometer welding head to perform welding according to a preset standard welding image, to obtain a plurality of welding image samples;

[0007] comparing and analyzing each welding image sample with the standard welding image to obtain welding data of the welding image sample, and labeling unqualified welding sections in the welding data to obtain labeled welding data;

[0008] obtaining welding process parameters of the unqualified welding sections in the welding image sample, and associating the welding process parameters with the welding data of the welding image sample to obtain training data;

[0009] constructing a visual detection model, and importing the training data into the visual detection model for training to obtain a trained visual detection model;

[0010] inputting the collected welding image to be detected into the trained visual detection model, the visual detection model judging whether the welding image to be detected has unqualified welding sections, and outputting a welding process parameter range of the unqualified welding sections, so that the galvanometer welding head can be adjusted in parameters according to the welding process parameter range of the unqualified welding sections.

[0011] In combination with the first aspect, in an implementable manner, the comparing and analyzing each of the welding image samples with the standard welding image to obtain welding data of the welding image sample comprises:

[0012] The welding image sample is smoothed to obtain a denoised welding image sample.

[0013] The welding area in the welding image sample is located through Blob analysis, and the welding area is subjected to second-order differential processing and Blob analysis to obtain welding data of the welding image sample, the welding data comprising at least one of a height of a welding segment and a width of the welding segment.

[0014] In combination with the first aspect, in an implementable manner, the labeling of the unqualified welding segment in the welding data to obtain labeled welding data comprises:

[0015] The welding data is subjected to first-level labeling, the label of the first-level labeling comprising one of qualified and unqualified, and the welding data is subjected to second-level labeling, the label of the second-level labeling comprising one of overlapping defect, dislocation defect and offset defect, to obtain the labeled welding data.

[0016] In combination with the first aspect, in an implementable manner, the obtaining of the welding process parameter of the unqualified welding segment in the welding image sample and the correlating of the welding process parameter with the welding data of the welding image sample comprise the following steps:

[0017] The first welding process parameter of the galvanometer welding head when welding the unqualified welding segment is obtained, and the first welding process parameter is correlated with the unqualified welding segment in the labeled welding image sample;

[0018] The second welding process parameter of the galvanometer welding head when welding a qualified welding segment corresponding to the position of the unqualified welding segment is obtained, and the second welding process parameter is correlated with the qualified welding segment in the labeled welding image sample.

[0019] In combination with the first aspect, in an implementable manner, the welding process parameter comprises at least one of a galvanometer welding head moving track, a galvanometer welding head speed, a galvanometer welding head acceleration, a galvanometer welding head laser power and a galvanometer welding head laser switch delay.

[0020] In combination with the first aspect, in a feasible implementation, the welding image to be detected and the welding image sample are acquired by an industrial camera, and the welding image to be detected and the welding image sample are both 2D gray images or 3D gray images, and the imaging device is a CCD industrial camera, a CMOS industrial camera, a 2D line scanning industrial camera or a 3D line scanning industrial camera.

[0021] In combination with the first aspect, in a feasible implementation, the visual detection model comprises at least one of a support vector machine, an artificial neural network and a random forest decision.

[0022] To achieve the above object, the third aspect provides a visual detection device, which comprises:

[0023] A control unit is configured to control the galvanometer welding head to perform welding according to a preset standard welding image, so as to obtain a plurality of welding image samples;

[0024] A comparison unit is configured to compare and analyze each welding image sample with the standard welding image, so as to obtain welding data of the welding image sample, and mark unqualified welding sections in the welding data, so as to obtain marked welding data;

[0025] An acquisition unit is configured to acquire welding process parameters of the unqualified welding sections in the welding image sample, and associate the welding process parameters with the welding data of the welding image sample, so as to obtain training data;

[0026] A training unit is configured to construct a visual detection model, and import the training data into the visual detection model for training until network parameters of the visual detection model converge, so as to obtain a trained visual detection model;

[0027] An output unit is configured to input the acquired welding image to be detected into the trained visual detection model, so that the visual detection model judges whether the welding image to be detected has unqualified welding sections, and outputs a welding process parameter range of the unqualified welding sections, so that the galvanometer welding head can be adjusted in parameters according to the welding process parameter range of the unqualified welding sections.

[0028] In combination with the second aspect, in a feasible implementation, the processing unit is further configured to:

[0029] Smooth the welding image sample to obtain a denoised welding image sample;

[0030] The welding region in the welding image sample is located by Blob analysis, and the welding region is subjected to second-order differential processing and Blob analysis, so as to obtain welding data of the welding image sample, the welding data including at least one of a height of a welding segment and a width of the welding segment.

[0031] To achieve the above-mentioned purpose, in a third aspect, the present application provides a readable storage medium, the storage medium comprising a stored program, which, when executed, controls a device in which the storage medium is located to perform the above-mentioned laser welding-based visual detection method.

[0032] In the present solution, the laser welding process parameters are associated with the welding data as training data to train the visual detection model, so that the visual detection model can identify qualified welding images or unqualified welding images, and then the trained visual detection model is used to detect the welding image to be detected. Since the visual detection model is trained by deep learning based on the labeled welding image sample, the visual detection model can learn the characteristics of various types of unqualified defects and the characteristics of unqualified welding process parameters according to the labeled labels, so that the visual detection model can identify whether the welding image to be detected is qualified, realize intelligent detection of the visual detection model, reduce the influence of different judgments due to personnel differences, improve the detection efficiency of the welding effect, and further improve the qualified rate of welding by outputting an optimization scheme for improving unqualified welding according to the welding process parameter range of the unqualified welding segment. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described in the following are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0034] Figure 1 is a flowchart of a laser welding-based visual detection method provided by an embodiment of the present application;

[0035] Figure 2-1 is a structural schematic diagram of a laser welding visual detection system provided by an embodiment of the present application;

[0036] Figure 2-2 is a working area schematic diagram of a laser galvanometer welding head in a laser welding visual detection system provided by an embodiment of the present application;

[0037] Figure 3 is a schematic diagram of an optional visual detection device provided by an embodiment of the present application;

[0038] Figure 4is a schematic diagram of an optional computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to better understand the technical solutions of the present application, the embodiments of the present application are described in detail below with reference to the drawings.

[0040] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0041] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0042] It should be understood that the term "and / or" used herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0043] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe the terminals, these terminals should not be limited to these terms. These terms are only used to distinguish the terminals from each other. For example, without departing from the scope of the embodiments of the present application, the first terminal can also be referred to as the second terminal, and similarly, the second terminal can also be referred to as the first terminal.

[0044] Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0045] Figure 1 is a flowchart of a visual detection method based on laser welding according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0046] In step S10, a plurality of welding image samples are obtained by controlling the galvanometer welding head to weld according to a preset standard welding image.

[0047] Step S20, each welding image sample is compared and analyzed with the standard welding image to obtain welding data of the welding image sample, and the unqualified welding section in the welding data is labeled to obtain labeled welding data.

[0048] Step S30, welding process parameters of the unqualified welding section in the welding image sample are obtained, and the welding process parameters are associated with the welding data of the welding image sample to obtain training data.

[0049] Step S40, a visual detection model is constructed, and the training data are imported into the visual detection model for training to obtain a trained visual detection model.

[0050] Step S50, the collected welding image to be detected is input into the trained visual detection model, the visual detection model judges whether the welding image to be detected has an unqualified welding section, and outputs a welding process parameter range of the unqualified welding section, so that the parameter of the galvanometer welding head can be adjusted according to the welding process parameter range of the unqualified welding section.

[0051] In the scheme, the laser welding process parameters are associated with the welding data as training data to train the visual detection model, so that the visual detection model can identify qualified welding images or unqualified welding images, and then the trained visual detection model is used to detect the welding image to be detected. Since the labeled welding image sample is used for deep learning training, the visual detection model can learn the characteristics of various unqualified defects and the characteristics of unqualified welding process parameters according to the labeled labels, so that the visual detection model can identify whether the welding image to be detected is qualified, realize intelligent detection of the visual detection model, reduce the influence of different personnel judgments, improve the detection efficiency of the welding effect, and further improve the qualified rate of welding by outputting an optimization scheme for improving unqualified welding according to the welding process parameter range of the unqualified welding section.

[0052] The above laser welding-based visual detection method will be described in detail in combination with specific embodiments as follows:

[0053] Step S10, the galvanometer welding head is controlled to weld according to a preset standard welding image to obtain a plurality of welding image samples.

[0054] In some embodiments, the standard welding image is a welding image designed by a customer or a designer according to a workpiece to be processed. Specifically, the standard welding image can be drawn by a drawing software such as CAD. The welding path in the labeled welding image can be a circle, a square, a circular array, a square array or other preset paths, which are not limited herein. Specifically, the line width of the welding section in the image drawn by CAD is within a preset range.

[0055] As shown in Figure 2-1 The laser galvanometer welding system includes a support 1, a laser galvanometer welding head 2, and a motion platform 3. The laser galvanometer welding head 2 is used to emit a laser beam to weld a workpiece. The laser galvanometer welding head 2 is connected to the support 1 and can deflect the laser beam to change the direction of the laser beam. As long as the workpiece is within the welding range of the laser galvanometer welding head, the motion platform 3 is used to move the laser galvanometer welding head to realize welding along a predetermined welding path.

[0056] The laser galvanometer welding head can make the spot formed by the laser beam on the surface of the workpiece swing along the predetermined welding path in a circular ring shape. That is, the spot first forms a circular ring when moving, and then moves along the predetermined welding path to form multiple circular rings.

[0057] The laser galvanometer welding system includes an imaging device 4 connected to the support 1. The imaging device 4 is used to obtain a to-be-detected welding image of the workpiece after welding. The to-be-detected welding image is a 2D grayscale image or a 3D grayscale image. The imaging device is a CCD industrial camera, a CMOS industrial camera, a 2D line-scan industrial camera, or a 3D line-scan industrial camera. In a specific embodiment, a CCD industrial camera is taken as an example for the following description. The distance between the CCD industrial camera and the laser galvanometer welding head can be calibrated by laser dotting the CCD industrial camera. Alternatively, the mechanical coordinate system of the CCD industrial camera is determined by a 9-point calibration method, and the camera distortion is calibrated by a correction plate. The welding image captured by the CCD industrial camera is a grayscale image with 5 million pixels.

[0058] For workpieces of the same specification, the welding path is constant, that is, the welding path input in advance into the laser galvanometer welding head will not change. However, for different workpieces, due to machining errors and other reasons, the position of the part to be welded may have slight deviations. In order to make the welding more accurate, it is necessary to determine the welding starting point for each workpiece, that is, the starting position of the spot formed on the surface of the workpiece after the laser beam passes through the laser galvanometer welding head. Of course, if the deviation is small, it can be ignored, and the imaging device can be omitted.

[0059] As shown in Figure 2-2 The rectangular area is a single welding working area of the laser galvanometer welding head. Due to the limited welding working area, when the welding working area exceeds the range, the motion platform can be used to move the laser galvanometer welding head, and then the welding is performed again. After multiple welding working areas are spliced, the complete welding path is completed. Therefore, at the junction of the welding working areas, mechanical displacement errors can easily cause defects such as overlapping, deviation, and dislocation at the interface.

[0060] In the embodiment, the standard welding image includes a welding path. After the welding path is determined, a welding simulation test can be performed. A plurality of welding image samples can be obtained through multiple weldings. Specifically, the welding image to be detected and the welding image samples can be acquired by an imaging device, and the welding image to be detected and the welding image samples are 2D gray images or 3D gray images. The imaging device is a CCD industrial camera, a CMOS industrial camera, a 2D line scanning industrial camera or a 3D line scanning industrial camera. Optionally, each preset standard welding image can correspond to at least 50 welding image samples. The specifications of each welding image sample should be uniform so that the welding image samples have better consistency.

[0061] Further, in order to improve the sample quantity, the welding image samples can also be augmented so as to diversify the samples. Specifically, the augmentation strategy can be at least one of rotation transformation, flip transformation, scaling transformation, dimension transformation, region cropping, noise addition, contrast transformation, color jittering and composite superposition.

[0062] It should be noted that the rotation transformation refers to randomly rotating the welding image by a preset angle to change the orientation of the welding image. The flip transformation refers to flipping the welding image along the horizontal or vertical direction. The scaling transformation refers to enlarging or reducing the image according to a preset scale. The dimension transformation refers to enlarging or reducing the image according to a preset dimension factor, or filtering the image to construct a dimension space according to the preset dimension factor, so as to change the size or blur degree of the image content. The region cropping refers to cropping a region of interest in the welding image. The noise addition refers to randomly superimposing some noise on the original picture. The contrast transformation refers to changing the saturation S and V brightness components in the HSV color space of the image while keeping the hue H unchanged, performing exponential operation (the exponential factor is between 0.25 and 4) on the S and V components of each pixel to increase the light change. The color jittering refers to randomly changing the exposure, saturation and hue of the image to form pictures under different illuminations and colors, so as to enable the model to use different illumination conditions as much as possible. The composite superposition refers to randomly extracting two pictures, processing the pictures through the basic data augmentation operation, superimposing and synthesizing a new sample through pixel average value, and taking one of the original sample labels as the label of the new sample.

[0063] Specifically, step S20 includes:

[0064] Step S21, each welding image sample is compared and analyzed with the standard welding image to obtain welding data of the welding image sample.

[0065] Step S22, the unqualified welding section in the welding data is labeled to obtain labeled welding data.

[0066] In step S21, the welding data of the welding image sample is obtained by comparing and analyzing the welding image sample with the standard welding image.

[0067] The welding image sample is smoothed to obtain a denoised welding image sample.

[0068] The welding region in the welding image sample is located by Blob analysis, and the welding region is subjected to second-order differential processing and Blob analysis to obtain welding data of the welding image sample, the welding data including at least one of a height of a welding segment and a width of the welding segment.

[0069] It should be noted that the smoothing processing includes at least one of Gaussian filtering processing, mean filtering processing, and bilateral filtering processing. Preferably, the welding image sample is subjected to the Gaussian filtering processing. The Blob analysis is an analysis of a connected domain of the same pixel (welding point, edge, etc.) in the welding image.

[0070] In other embodiments, other analysis means can also be used to analyze the welding image, as long as the welding data of the welding path in the welding image can be obtained. When the imaging device is a 2D industrial camera, the welding data can be the width of the welding segment. When the imaging device is a three-dimensional industrial camera, the welding data can be the width and height of the welding segment.

[0071] It should be noted that after the second-order differential processing, the starting point of the welding path is taken as a starting point of data collection of the welding data, and data collection is performed. In this embodiment, the welding data is a set of shortest distances from a differential point at one end of a current welding segment to another welding segment adjacent to the current welding segment. In order to improve the calculation efficiency of the welding data, in some embodiments, high-concurrency processing based on an Open Computing Language (OpenCL) software library can be used to speed up the algorithm, and the computing power of a high-performance graphics processing unit (GPU) can be increased.

[0072] Further, in step S22, unqualified welding segments in the welding data are labeled to obtain labeled welding data, specifically including the following steps:

[0073] The welding data is subjected to first-level labeling, and the label of the first-level labeling includes one of qualified and unqualified; and the welding data is subjected to second-level labeling, and the label of the second-level labeling includes one of an overlap defect, a dislocation defect, and a misalignment defect, to obtain the labeled welding data.

[0074] The welding data of each welding image sample is provided with a label, for example, welding data with a positive label, i.e., welding data (welding image sample) marked as qualified, welding data with a negative label, i.e., welding data (welding image sample) marked as unqualified. The unqualified label further includes a sub-label, which specifically includes one of an overlap defect, a dislocation defect, and a misalignment defect. When setting the label, (A1) can be used to represent the label of the welding data, where A is a first-level label indicating qualified or unqualified, and 1 is a second-level label indicating an overlap defect. Therefore, the welding data of the welding image sample is unqualified due to the overlap defect.

[0075] In step S30, welding process parameters of the unqualified welding section in the welding image sample are obtained, and the welding process parameters are associated with the welding data of the welding image sample to obtain training data.

[0076] Specifically, the first welding process parameters of the galvanometer welding head when welding the unqualified welding section are obtained, and the first welding process parameters are associated with the unqualified welding section in the labeled welding image sample.

[0077] The second welding process parameters of the galvanometer welding head when welding the qualified welding section corresponding to the position of the unqualified welding section are obtained, and the second welding process parameters are associated with the qualified welding section in the labeled welding image sample.

[0078] In some embodiments, the welding process parameters include at least one of a galvanometer welding head movement trajectory, a galvanometer welding head speed, a galvanometer welding head acceleration, and a galvanometer welding head power. Associating the welding process parameters with the labeled welding data can clearly record the welding process parameters of the unqualified welding section in the welding image. In the welding process of the same specification of workpiece, the welding process parameters of the qualified welding section at the same position can be used as an optimization scheme to optimize the unqualified welding section. For example, the speed of the galvanometer welding head when welding the welding section (L1) is between V1 and V2. When the speed of the galvanometer welding head is greater than V2 or less than V1, the welding section will have a dislocation defect or a misalignment defect.

[0079] It can be understood that by associating the welding process parameters with the welding data, the visual detection model can extract the welding process parameters in the subsequent training process, so as to give the welding process parameter range causing such unqualified defects. For example, in the 50 welding data with overlapping defect unqualified labels, after obtaining the welding process parameters associated with the 50 welding data, it is found through analysis that the speed of the galvanometer welding head is between (V3, V4), and in the second welding process parameters of the qualified welding section corresponding to the unqualified welding section position, the speed of the galvanometer welding head is between (V1, V2), so it can be found that the speed of the galvanometer welding head deviates from the preset range, and the overlapping defect is easy to occur.

[0080] In step S40, a visual detection model is constructed, and the training data is imported into the visual detection model for training to obtain a trained visual detection model.

[0081] Optionally, the visual detection model includes at least one of a support vector machine, an artificial neural network, and a random forest decision. Preferably, the visual detection model is a support vector machine model.

[0082] In the specific training process, the support vector machine algorithm is used to learn and classify the training data to obtain the model parameters of the trained visual detection model.

[0083] In step S50, the collected welding image to be detected is input into the trained visual detection model, the visual detection model judges whether the welding image to be detected has an unqualified welding section, and outputs the welding process parameter range of the unqualified welding section, so that the galvanometer welding head can be adjusted according to the welding process parameter range of the unqualified welding section.

[0084] In some embodiments, the welding image to be detected can be collected by an imaging device. Similarly, before inputting into the visual detection model, the welding image to be detected also needs to be smoothed to obtain a denoised welding image to be detected; then the welding area in the welding image to be detected is located through Blob analysis, and the welding area is subjected to second-order differential processing and Blob analysis to obtain welding data of the welding image to be detected, the welding data including at least one of the height of the welding section and the width of the welding section.

[0085] The visual detection model can extract the features of the welding data and judge whether the welding data is qualified or unqualified based on the features. In some embodiments, the data output by the visual detection model includes whether it is qualified and the probability of the defect it belongs to.

[0086] Step S50 further includes:

[0087] According to the label type of the unqualified welding section, all welding process parameters associated with the welding data of the same type of label stored in the model are obtained.

[0088] According to all welding process parameters associated with the welding data of the same type label, feature extraction is performed to obtain the welding process parameter range of the unqualified welding section.

[0089] Exemplarily, when the welding image to be detected is unqualified due to a misalignment defect, the visual detection model outputs all welding process parameters of the misalignment defect label stored after training according to the misalignment defect label; so that the user can adaptively adjust the specific welding process parameters according to the welding process parameter range output by the model, so as to eliminate the misalignment defect.

[0090] The embodiment of the application provides a visual detection device for executing the above-mentioned visual detection method based on laser welding, as shown in the figure, the device comprises a control unit 10, a processing unit 20, an acquisition unit 30, a training unit 40 and an output unit 50. Figure 3

[0091] The control unit 10 is used for controlling the galvanometer welding head to weld according to a preset standard welding image, so as to obtain a plurality of welding image samples;

[0092] The processing unit 20 is used for comparing and analyzing each welding image sample with the standard welding image, obtaining welding data of the welding image sample, and labeling unqualified welding sections in the welding data to obtain labeled welding data;

[0093] The acquisition unit 30 is used for acquiring welding process parameters of the unqualified welding section in the welding image sample, and associating the welding process parameters with the welding data of the welding image sample to obtain training data;

[0094] The training unit 40 is used for constructing a visual detection model, and importing the training data into the visual detection model for training until the network parameters of the visual detection model converge, so as to obtain a trained visual detection model;

[0095] The output unit 50 is used for inputting the collected welding image to be detected into the trained visual detection model, the visual detection model judges whether the welding image to be detected has an unqualified welding section, and outputs a welding process parameter range of the unqualified welding section, so that the galvanometer welding head can be adjusted in parameters according to the welding process parameter range of the unqualified welding section.

[0096] ​In the scheme, the laser welding process parameters are associated with the welding data as training data to train the visual detection model, so that the visual detection model can identify qualified welding images or unqualified welding images. The trained visual detection model is used to detect the welding images to be detected. Due to the deep learning training according to the labeled welding image samples, the visual detection model can learn the characteristics of various unqualified defects and the characteristics of unqualified welding process parameters according to the labeled labels, so that the visual detection model can identify whether the welding image to be detected is qualified, realize intelligent detection of the visual detection model, reduce the influence of different judgments due to personnel differences, improve the detection efficiency of the welding effect, and also output an optimization scheme for improving the unqualified welding according to the welding process parameter range of the unqualified welding section, so as to further improve the qualified rate of welding.

[0097] In some embodiments, the standard welding image is a welding image designed by a customer or a designer according to a workpiece to be processed. Specifically, the standard welding image can be drawn by drawing software such as CAD. The welding path in the labeled welding image can be a circle, a square, a circular array, a square array, or other preset paths, which are not limited herein. Specifically, the line width of the welding section in the CAD image is within a preset range.

[0098] In the embodiment, the standard welding image includes a welding path. After the welding path is determined, a welding simulation test can be performed. A plurality of welding image samples are obtained by multiple welding. Specifically, the welding image to be detected and the welding image samples can be collected by an imaging device, and the welding image to be detected and the welding image samples are both 2D gray images or 3D gray images. The imaging device is a CCD industrial camera, a CMOS industrial camera, a 2D line scanning industrial camera, or a 3D line scanning industrial camera. Optionally, each preset standard welding image corresponds to at least 50 welding image samples. The specifications of each welding image sample should be uniform.

[0099] Further, in order to improve the sample size, the welding image samples can also be augmented to diversify the samples. Specifically, the augmentation strategy can be at least one of rotation transformation, flip transformation, scaling transformation, scaling transformation, region cropping, noise addition, contrast transformation, color jitter, and composite superposition.

[0100] It should be noted that the rotation transformation refers to rotating the preset angle of the welding image randomly to change the orientation of the welding image. The flip transformation refers to flipping the welding image along the horizontal or vertical direction. The scaling transformation refers to enlarging or reducing the image according to the preset scale. The scale transformation refers to enlarging or reducing the image according to the preset scale factor, or filtering the image to construct a scale space according to the preset scale factor, to change the size or blur degree of the image content. The region cropping refers to cropping the region of interest in the welding image. The noise adding refers to randomly superimposing some noise on the original picture. The contrast transformation refers to changing the saturation S and V brightness components in the HSV color space of the image, keeping the hue H unchanged, and performing exponential operation (the exponential factor is between 0.25 and 4) on the S and V components of each pixel to increase the light change. The color jittering refers to randomly changing the exposure, saturation and hue of the image to form pictures under different light and color conditions, so as to make the model as much as possible to use the case under different light conditions. The composite superposition refers to randomly extracting two pictures, respectively processing them through the basic data augmentation operation, and then superimposing and synthesizing a new sample in the form of pixel average value, and the label of the new sample is one of the labels of the original samples.

[0101] Specifically, the processing unit 20 includes an analysis subunit and a labeling subunit.

[0102] The analysis subunit is configured to compare and analyze each welding image sample with a standard welding image to obtain welding data of the welding image sample.

[0103] The labeling subunit is configured to label unqualified welding sections in the welding data to obtain labeled welding data.

[0104] The analysis subunit is further configured to perform smoothing processing on the welding image sample to obtain a denoised welding image sample, locate a welding area in the welding image sample through Blob analysis, and perform second-order differential processing and Blob analysis on the welding area to obtain the welding data of the welding image sample. In some embodiments, the welding data includes at least one of a height of a welding section and a width of the welding section.

[0105] It should be noted that the smoothing processing includes at least one of Gaussian filtering processing, mean filtering processing and bilateral filtering processing. Preferably, the Gaussian filtering processing is used to perform smoothing processing on the welding image sample. The Blob analysis is an analysis on a connected domain of the same pixel (welding point, edge, etc.) in the welding image.

[0106] In other embodiments, other analysis means can also be used to analyze the welding image, as long as the welding data of the welding path in the welding image can be obtained. When the imaging device is a 2D industrial camera, the welding data can be the width of the welding segment. When the imaging device is a three-dimensional industrial camera, the welding data can be the width and height of the welding segment.

[0107] It should be noted that after the second-order differential processing, the starting point of the welding path is taken as the starting point of the collection of the welding data, and the data collection is performed. In this embodiment, the welding data is a set of the shortest distances from one end of the current welding segment to the adjacent other welding segment. In order to improve the calculation efficiency of the welding data, in some embodiments, high-concurrency processing based on the OpenCL software library can be used to speed up the algorithm, and the computing power can be increased by using a high-performance GPU.

[0108] Further, the labeling unit 22 is specifically configured to perform first-level labeling on the welding data, and the first-level label includes one of qualified and unqualified; and perform second-level labeling on the welding data, and the second-level label includes one of overlapping defect, dislocation defect and offset defect, to obtain the labeled welding data.

[0109] The welding data of each welding image sample is provided with a label, for example, welding data with a positive label, that is, welding data (welding image sample) marked as qualified, and welding data with a negative label, that is, welding data (welding image sample) marked as unqualified. The unqualified label further includes a sub-label, and the sub-label specifically includes one of overlapping defect, dislocation defect and offset defect. When setting the label, (A1) can be used to represent the label of the welding data, wherein A is the first-level label, that is, qualified or unqualified; 1 is the second-level label, that is, overlapping defect; and then the welding data of the welding image sample is unqualified due to the overlapping defect.

[0110] Further, the acquisition unit 30 is configured to:

[0111] acquire the first welding process parameter of the galvanometer welding head when welding the unqualified welding segment, and associate the first welding process parameter with the unqualified welding segment in the labeled welding image sample;

[0112] acquire the second welding process parameter of the galvanometer welding head when welding the qualified welding segment corresponding to the position of the unqualified welding segment, and associate the second welding process parameter with the qualified welding segment in the labeled welding image sample.

[0113] In some embodiments, the welding process parameters include at least one of a mirror welding head moving trajectory, a mirror welding head speed, a mirror welding head acceleration, and a mirror welding head power. By associating the welding process parameters with the labeled welding data, the welding process parameters of the unqualified welding section in the welding image can be clearly recorded. In the welding process of the same type of workpiece, the welding process parameters of the qualified welding section at the same position can be used as an optimization scheme for optimizing the unqualified welding section. For example, the mirror welding head speed is between V1 and V2 when welding the welding section (L1). When the mirror welding head speed is greater than V2 or less than V1, the welding section will have a dislocation defect or a deviation defect.

[0114] It can be understood that by associating the welding process parameters with the welding data, the visual detection model can extract the welding process parameters in the subsequent training process, so as to give the welding process parameter range that causes such unqualified defects. For example, in 50 welding data with overlapping defect unqualified labels, after obtaining the welding process parameters associated with the 50 welding data, it is found through analysis that the mirror welding head speed is between (V3, V4), and in the second welding process parameters of the qualified welding section corresponding to the unqualified welding section position, the mirror welding head speed is between (V1, V2), so it can be found that the mirror welding head speed deviates from the preset range, which is easy to produce overlapping defects.

[0115] Optionally, the visual detection model includes at least one of a support vector machine, an artificial neural network, and a random forest decision. Preferably, the visual detection model is a support vector machine model.

[0116] In a specific training process, the support vector machine algorithm is used to learn and classify the training data to obtain the model parameters of the trained visual detection model.

[0117] In some embodiments, the imaging device can be used to acquire the welding image to be detected. Similarly, before inputting the visual detection model, the welding image to be detected also needs to be smoothed to obtain a denoised welding image to be detected; then the welding area in the welding image to be detected is located through Blob analysis, and the welding area is subjected to second-order differential processing and Blob analysis to obtain the welding data of the welding image to be detected, which includes at least one of the height of the welding section and the width of the welding section.

[0118] The visual detection model can extract the features of the welding data and judge whether the welding data is qualified or unqualified based on the features. In some embodiments, the data output by the visual detection model includes whether it is qualified and the probability of belonging to a defect.

[0119] The output unit 50 is also used for:

[0120] According to the label type of the unqualified welding section, all welding process parameters associated with the welding data of the same type of label stored in the model are obtained;

[0121] According to all welding process parameters associated with the welding data of the same type of label, feature extraction is performed to obtain the welding process parameter range of the unqualified welding section.

[0122] Exemplarily, when the welding image to be detected is caused by a misalignment defect, the visual detection model stores all welding process parameters of the misalignment defect label after training according to the misalignment defect label, so that the user can adaptively adjust the specific welding process parameters according to the welding process parameter range output by the model to eliminate the misalignment defect.

[0123] The embodiment of the present application provides a readable storage medium, the storage medium comprises a stored program, wherein when the program runs, the device where the storage medium is located is controlled to perform the following steps:

[0124] According to the preset standard welding image, the galvanometer welding head is controlled to weld to obtain a plurality of welding image samples; each welding image sample is compared and analyzed with the standard welding image to obtain welding data of the welding image sample, and the unqualified welding section in the welding data is labeled to obtain labeled welding data; the welding process parameters of the unqualified welding section in the welding image sample are obtained, and the welding process parameters are associated with the welding data of the welding image sample to obtain training data; a visual detection model is constructed, and the training data is imported into the visual detection model for training to obtain a trained visual detection model; the collected welding image to be detected is input into the trained visual detection model, the visual detection model judges whether the welding image to be detected has an unqualified welding section, and outputs the welding process parameter range of the unqualified welding section, so that the galvanometer welding head can be adjusted in parameters according to the welding process parameter range of the unqualified welding section.

[0125] Optionally, when the program runs, the device where the storage medium is located is controlled to perform the following steps: the welding image sample is smoothed to obtain a denoised welding image sample; the welding area in the welding image sample is located by Blob analysis, and the welding area is subjected to second-order differential processing and Blob analysis to obtain the welding data of the welding image sample, the welding data including at least one of the height of the welding section and the width of the welding section.

[0126] Optionally, when the program runs, the device where the storage medium is located is controlled to perform the following steps: the welding data is subjected to first-level labeling, and the label of the first-level labeling includes one of qualified and unqualified; and the welding data is subjected to second-level labeling, and the label of the second-level labeling includes one of overlapping defect, dislocation defect and misalignment defect, to obtain the labeled welding data.

[0127] Optionally, the device where the storage medium is located performs the following steps when the program is running: obtaining a first welding process parameter of the galvanometer welding head when welding a substandard welding section, and associating the first welding process parameter with the substandard welding section in the labeled welding image sample;

[0128] obtaining a second welding process parameter of the galvanometer welding head when welding a qualified welding section corresponding to the position of the substandard welding section, and associating the second welding process parameter with the qualified welding section in the labeled welding image sample.

[0129] Figure 4 is a schematic diagram of a computer device provided by an embodiment of the present application. As shown in the figure, the computer device 100 of the embodiment includes a processor 101, a memory 102, and a computer program 103 stored in the memory 102 and executable on the processor 101. When the processor 101 executes the computer program 103, the method for measuring the weight of live pigs in the embodiment is implemented, and to avoid repetition, details are not described here. Alternatively, when the computer program is executed by the processor 101, the functions of each model / unit in the live pig weight measurement device in the embodiment are implemented, and to avoid repetition, details are not described here. Figure 4

[0130] The computer device 100 can be a desktop computer, a notebook, a palm computer, and a cloud server, etc. The computer device can include, but is not limited to, the processor 101 and the memory 102. Those skilled in the art can understand that Figure 3 The computer device 100 is only an example and does not constitute a limitation on the computer device 100, and can include more or fewer components than the figure, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0131] The processor 101 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0132] ​The memory 102 can be an internal storage unit of the computer device 100, such as a hard disk or a memory of the computer device 100. The memory 102 can also be an external storage device of the computer device 100, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the memory 102 can include both an internal storage unit and an external storage device of the computer device 100. The memory 102 is used to store computer programs and other programs and data required by the computer device. The memory 102 can also be used to temporarily store data that has been output or is to be output.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0134] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are merely schematic, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0135] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0136] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0137] The integrated units in the form of software function units can be stored in a computer readable storage medium. The software function units are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0138] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in various embodiments of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units, modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional modules, sub-modules and units according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0140] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for visual inspection based on laser welding, characterized in that, The method comprises: controlling a galvanometer welding head to weld according to a preset standard welding image to obtain a plurality of welding image samples; comparing and analyzing each welding image sample with the standard welding image to obtain welding data of the welding image sample, and labeling unqualified welding sections in the welding data to obtain labeled welding data; obtaining welding process parameters of the unqualified welding sections in the welding image sample, and associating the welding process parameters with the welding data of the welding image sample to obtain training data; constructing a visual detection model, and importing the training data into the visual detection model for training to obtain a trained visual detection model; inputting a collected welding image to be detected into the trained visual detection model, the visual detection model judging whether the welding image to be detected has unqualified welding sections, and outputting welding process parameter ranges of the unqualified welding sections to enable parameter adjustment of the galvanometer welding head according to the welding process parameter ranges of the unqualified welding sections; wherein the labeling of the unqualified welding sections in the welding data to obtain labeled welding data comprises: first-level labeling of the welding data, the labels of the first-level labeling including one of qualified and unqualified; and second-level labeling of the welding data, the labels of the second-level labeling including one of overlapping defects, dislocation defects and offset defects to obtain labeled welding data; the obtaining of the welding process parameters of the unqualified welding sections in the welding image sample and the associating of the welding process parameters with the welding data of the welding image sample comprise the following steps: obtaining first welding process parameters of the galvanometer welding head when welding the unqualified welding sections, and associating the first welding process parameters with the unqualified welding sections in the labeled welding image sample; obtaining second welding process parameters of the galvanometer welding head when welding qualified welding sections corresponding to positions of the unqualified welding sections, and associating the second welding process parameters with the qualified welding sections in the labeled welding image sample. the comparing and analyzing of each welding image sample with the standard welding image to obtain welding data of the welding image sample comprises:

2. The method of claim 1, wherein, smoothing the welding image sample to obtain a denoised welding image sample; locating a welding area in the welding image sample through Blob analysis, and performing second-order differential processing and Blob analysis on the welding area to obtain welding data of the welding image sample, the welding data including at least one of a height of a welding section and a width of a welding section. The welding process parameters include at least one of a galvanometer welding head movement trajectory, a galvanometer welding head speed, a galvanometer welding head acceleration and a galvanometer welding head power.

3. The method of claim 1, wherein, ​ 4. The method of claim 1, wherein, The welding image to be detected and the welding image sample are acquired by an imaging device, and the welding image to be detected and the welding image sample are both 2D gray images or 3D gray images, and the imaging device is a CCD industrial camera, a CMOS industrial camera, a 2D line scanning industrial camera or a 3D line scanning industrial camera.

5. The method according to any one of claims 1 to 4, characterized in that, The visual detection model comprises at least one of a support vector machine, an artificial neural network and a random forest decision.

6. A vision inspection apparatus characterized by comprising: The device comprises: a control unit configured to control the galvanometer welding head to perform welding according to a preset standard welding image, so as to obtain a plurality of welding image samples; a processing unit configured to compare and analyze each welding image sample with the standard welding image, so as to obtain welding data of the welding image sample, and mark unqualified welding sections in the welding data, so as to obtain marked welding data; an acquisition unit configured to acquire welding process parameters of the unqualified welding sections in the welding image sample, and associate the welding process parameters with the welding data of the welding image sample, so as to obtain training data; a training unit configured to construct a visual detection model, and import the training data into the visual detection model for training until network parameters of the visual detection model converge, so as to obtain a trained visual detection model; an output unit configured to input the acquired welding image to be detected into the trained visual detection model, so that the visual detection model judges whether the welding image to be detected has unqualified welding sections, and outputs welding process parameter ranges of the unqualified welding sections, so that the galvanometer welding head can be adjusted in parameters according to the welding process parameter ranges of the unqualified welding sections; the marking of the unqualified welding sections in the welding data to obtain the marked welding data comprises: first-level marking of the welding data, wherein labels of the first-level marking comprise one of qualified and unqualified; and second-level marking of the welding data, wherein labels of the second-level marking comprise one of overlapping defects, dislocation defects and offset defects, so as to obtain the marked welding data; the acquisition of the welding process parameters of the unqualified welding sections in the welding image sample and the association of the welding process parameters with the welding data of the welding image sample comprise the following steps: acquiring first welding process parameters of the galvanometer welding head when welding the unqualified welding sections, and associating the first welding process parameters with the unqualified welding sections in the marked welding image sample; acquiring second welding process parameters of the galvanometer welding head when welding qualified welding sections corresponding to positions of the unqualified welding sections, and associating the second welding process parameters with the qualified welding sections in the marked welding image sample. The processing unit is further configured to:

7. The apparatus of claim 6, wherein, smooth the welding image sample to obtain a denoised welding image sample; locate a welding area in the welding image sample through Blob analysis, and perform second-order differential processing and Blob analysis on the welding area, so as to obtain welding data of the welding image sample, wherein the welding data comprises at least one of a height of a welding section and a width of the welding section. ​ 8. A readable storage medium, the storage medium comprising a stored program, characterized in that, The program controls the device where the storage medium is located to execute the visual inspection method based on laser welding according to any one of claims 1 to 5 when the program is running.

Images