Weld quality detection method and training method

Abstract

The application provides a weld quality detection method and a training method. The weld quality detection method comprises the following steps: obtaining a first gray-scale image, wherein the first gray-scale image is obtained by sequentially irradiating a detected object by at least two light sources, the number of the first gray-scale image corresponds to the number of the light sources, and the light sources output parallel light. A first normal angle image of the detected object is obtained based on the first gray-scale image. The first normal angle image is divided into a marked area. And the first normal angle image and the division result of the marked area are output and / or displayed to obtain a weld quality detection result. More information is obtained based on the first normal angle image, and the information itself is more accurate, which can improve the detection accuracy of the weld quality detection, thereby solving the problem of high false rejection rate and high false acceptance rate in the prior art when the weld quality of a measured object with high surface reflectivity is detected.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method for weld quality inspection and a training method. Background Technology

[0002] In existing technologies, the inspection of weld surface quality mainly relies on laser stereo imaging or binocular stereo vision. The former uses laser structured light for scanning, while the latter obtains three-dimensional information of the weld surface based on the principle of parallax, and identifies defects in the weld surface height map or three-dimensional point cloud. This three-dimensional reconstruction technology is usually time-consuming, especially when the surface of the object being measured has high reflectivity, and many areas are prone to being unmeasurable. The three-dimensional point cloud has high sparsity, and when the weld size is small, the amount of effective data is limited. These drawbacks result in a high false positive and false negative rate for the related methods when inspecting the surface quality of welds made of highly reflective materials.

[0003] In other words, existing technologies suffer from high false positive and false negative rates when inspecting weld quality on objects with high surface reflectivity. Summary of the Invention

[0004] The purpose of this invention is to provide a weld quality inspection method and training method to solve the problem of high false positive and false negative rates when inspecting weld quality on objects with high surface reflectivity in the prior art.

[0005] To address the aforementioned technical problems, according to a first aspect of the present invention, a weld quality inspection method is provided, comprising the following steps: acquiring a first grayscale image, wherein the first grayscale image is obtained by sequentially illuminating an object under inspection with at least two light sources, the number of first grayscale images corresponding to the number of light sources, and the light sources outputting parallel light; acquiring a first normal angle image of the object under inspection based on the first grayscale image; dividing a marked area based on the first normal angle image; and outputting and / or displaying the first normal angle image and the division result of the marked area to obtain a weld quality inspection result.

[0006] Optionally, the step of obtaining the first normal angle map of the inspected object based on the first grayscale image includes: obtaining the first normal angle map based on a mapping model and the first grayscale image; wherein, the method of obtaining the mapping model includes the following steps: the at least two light sources sequentially illuminate the first sample to obtain a second grayscale image, wherein the number of second grayscale images corresponds to the number of light sources, and the second normal angle map of the first sample is known; and, obtaining the mapping model based on the second normal angle map and the second grayscale image.

[0007] Optionally, the step of obtaining the first normal angle map based on the mapping model and the first grayscale image includes: if the grayscale of a pixel in each of the first grayscale images meets the hole judgment condition, the normal angle corresponding to the pixel is set to 0°; and if the grayscale of a pixel in any of the first grayscale images does not meet the hole judgment condition, the normal angle corresponding to the pixel is obtained based on the mapping model.

[0008] Optionally, the first normal angle image is stored and displayed in grayscale. The step of obtaining the first normal angle image based on the mapping model and the first grayscale image further includes: the grayscale value of the pixel in the first normal angle image is converted and calculated according to the normal angle corresponding to the pixel, wherein 180° is converted into the maximum grayscale value, 0° is converted into the minimum grayscale value, and the other angles are converted according to the ratio.

[0009] Optionally, the angle between the illumination direction of the light source and the illuminated plane of the object under inspection is less than a preset angle.

[0010] Optionally, the weld quality inspection method further includes the following steps: acquiring an illumination color image of the inspected object under preset conditions, wherein the pixels on the illumination color image and the first normal angle image have a corresponding relationship; and outputting and / or displaying the first normal angle image, the illumination color image, and the division result of the marked area to obtain the weld quality inspection result, wherein the weld quality inspection result includes the surface unevenness of the inspected object.

[0011] Optionally, the marked region includes a horizontal region, and the step of dividing the marked region based on the first normal angle map includes: merging pixels whose normal angles belong to the horizontal determination interval to obtain at least one first sub-region; and, if the first sub-region is larger than a first preset area, the first sub-region is divided into the horizontal region.

[0012] Optionally, the marked region includes a hole region, and the step of dividing the marked region based on the first normal angle map includes: merging pixels that meet the hole determination conditions to obtain at least one second sub-region; and if the second sub-region is larger than a second preset area, the second sub-region is divided into the hole region, or the second sub-region is expanded based on a preset rule, and the expanded second sub-region is divided into the hole region.

[0013] Optionally, the marked region includes a horizontal region, a hole region, and a non-horizontal region. The step of dividing the marked region based on the first normal angle map includes: dividing the horizontal region and the hole region; clustering pixels not divided into the horizontal region and the hole region to obtain at least one class of third sub-regions; configuring the largest area in each class of third sub-regions as a candidate sub-region; and if the area of ​​the candidate sub-region is greater than a third preset area and the drop height of the candidate sub-region is greater than a preset height, the candidate sub-region is divided into the non-horizontal region.

[0014] To address the aforementioned technical problems, according to a second aspect of the present invention, a training method is provided for training a weld quality inspection model. The training method includes the following steps: acquiring a first grayscale image, wherein the first grayscale image is obtained by sequentially illuminating a second sample with at least two light sources, the number of first grayscale images corresponding to the number of light sources, and the light sources output parallel light; acquiring a first normal angle image of the inspected object based on the first grayscale image; acquiring an illumination color image of the second sample under preset conditions, wherein the pixels on the illumination color image and the first normal angle image have a corresponding relationship; dividing a marked region based on the first normal angle image; and training the weld quality inspection model based on the first normal angle image, the illumination color image, and the division result of the marked region; wherein the weld quality inspection model is used to output a weld quality inspection result based on the illumination color image of the inspected object.

[0015] Compared with existing technologies, the weld quality inspection method and training method provided by this invention include the following steps: acquiring a first grayscale image, wherein the first grayscale image is obtained by sequentially illuminating the inspected object with at least two light sources, the number of first grayscale images corresponding to the number of light sources, and the light sources output parallel light; acquiring a first normal angle image of the inspected object based on the first grayscale image; dividing a marked area based on the first normal angle image; and outputting and / or displaying the first normal angle image and the division result of the marked area to obtain a weld quality inspection result. The first normal angle image provides more information, and the information it carries is more accurate, which can improve the detection accuracy of weld quality inspection, thereby solving the problem of high false positive and false negative rates when performing weld quality inspection on objects with high surface reflectivity in existing technologies. Attached Figure Description

[0016] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0017] Figure 1This is a schematic flowchart of a weld quality inspection method according to an embodiment of the present invention;

[0018] Figure 2a This is a schematic diagram illustrating the working principle of a normal angle diagram according to an embodiment of the present invention. Figure 1 ;

[0019] Figure 2b This is a schematic diagram of the working principle of the normal angle diagram according to an embodiment of the present invention (II).

[0020] Figure 3a This is a schematic diagram of a convex surface according to an embodiment of the present invention;

[0021] Figure 3b This is a schematic diagram of a concave surface according to an embodiment of the present invention;

[0022] Figure 4a This is a schematic diagram of the horizontal region according to an embodiment of the present invention;

[0023] Figure 4b This is a schematic diagram of the hole area according to an embodiment of the present invention;

[0024] Figure 4c This is a schematic diagram of a non-horizontal region according to an embodiment of the present invention;

[0025] Figure 5 This is a flowchart illustrating a training method according to an embodiment of the present invention.

[0026] In the attached image:

[0027] 1-Imaging device; 2-Light source; 3-Object under inspection; 11-First orientation angle diagram; 12-Division diagram; 13-Color illumination diagram; 14-Horizontal area; 15-Pore area; 16-Non-horizontal area. Detailed Implementation

[0028] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to facilitate and clarify the explanation of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.

[0029] As used in this invention, the singular forms “a,” “an,” and “the” include plural objects; the term “or” is generally used to mean “and / or”; the term “a number” is generally used to mean “at least one”; and the term “at least two” is generally used to mean “two or more”. Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first,” “second,” or “third” may explicitly or implicitly include one or at least two of that feature. “One end” and “the other end,” as well as “proximal end” and “distal end,” generally refer to two corresponding parts, including not only endpoints. The terms “installed,” “connected,” and “joined” should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral part; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements or an interaction between two elements. Furthermore, as used in this invention, the phrase "one element is disposed on another element" generally only indicates that there is a connection, coupling, cooperation, or transmission relationship between the two elements, and the connection, coupling, cooperation, or transmission between the two elements can be direct or indirect through an intermediate element. It should not be construed as indicating or implying a spatial positional relationship between the two elements, i.e., one element can be located arbitrarily inside, outside, above, below, or to one side of the other element, unless otherwise explicitly stated. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0030] The core idea of ​​this invention is to provide a weld quality inspection method and training method to solve the problem of high false positive and false negative rates when inspecting weld quality of objects with high surface reflectivity in the prior art.

[0031] The following description refers to the accompanying drawings. Please refer to them. Figures 1 to 5 ,in, Figure 1 This is a schematic flowchart of a weld quality inspection method according to an embodiment of the present invention; Figure 2a This is a schematic diagram illustrating the working principle of a normal angle diagram according to an embodiment of the present invention. Figure 1 ; Figure 2b This is a schematic diagram of the working principle of the normal angle diagram according to an embodiment of the present invention (II). Figure 3a This is a schematic diagram of a convex surface according to an embodiment of the present invention; Figure 3b This is a schematic diagram of a concave surface according to an embodiment of the present invention; Figure 4a This is a schematic diagram of the horizontal region according to an embodiment of the present invention; Figure 4b This is a schematic diagram of the hole area according to an embodiment of the present invention; Figure 4c This is a schematic diagram of a non-horizontal region according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating a training method according to an embodiment of the present invention.

[0032] This embodiment provides a weld quality inspection method, which features high speed, good stability, and high repeatability. It is particularly advantageous for inspected objects with high reflectivity, but can also be applied to inspected objects with low reflectivity. In this specification, the concept of "high reflectivity" should be understood based on common knowledge in the art, and can also be understood as a reflectivity higher than a preset reflectivity.

[0033] like Figure 1 As shown, the weld quality inspection method includes the following steps:

[0034] S10 acquires a first grayscale image, wherein the first grayscale image is obtained based on at least two light sources sequentially illuminating the object under inspection, the number of first grayscale images corresponds to the number of light sources, and the light sources output parallel light.

[0035] S20 obtains the first normal angle map of the inspected object based on the first grayscale image.

[0036] S30 divides the marked area based on the first normal angle diagram.

[0037] Additionally, S40 outputs and / or displays the first normal angle diagram and the division result of the marked area to obtain the weld quality inspection result.

[0038] In step S10, the relative positional relationship between the object under inspection and the imaging device remains unchanged during each first grayscale image capture. "Sequential illumination" should be understood as the light sources not being turned on simultaneously to prevent mutual interference. The imaging device is preferably a telecentric lens with a lens size larger than the object under inspection; this configuration can ignore grayscale differences caused by the pinhole imaging principle. "Parallel light" should be understood as parallel within an engineering scope, meeting engineering application standards, but not requiring parallelism under laboratory precision. "At least two light sources" should be distinguished by their relative illumination positions. If there exists a scheme using only one physical light source, but moved by a positioning device to illuminate the object under inspection from at least two preset positions, or if the object under inspection and the imaging device are moved by a positioning device so that the physical light source can illuminate the object under inspection from at least two preset positions, these variations should also be understood as "at least two light sources."

[0039] In step S20, the first normal angle map refers to a map in which each pixel represents the normal angle at its corresponding position. However, the specific data format can be arbitrarily chosen. A specific data setting method for the first normal angle map will be introduced later. It is understandable that if another scheme exists, using the tangent of each pixel as reference data for analysis, the tangent and normal can be converted to each other. If a "tangent angle map" appears in another scheme, it can also be understood as a generalized "normal angle map" because the conversion between the two is very simple and direct. It is understandable that the above-mentioned variations, or other schemes obtained by simply converting the normal angle, can all be broadly understood as "normal angle maps" if each pixel still contains normal angle information.

[0040] In step S40, if the solution adopts an output method, it is equivalent to inputting the data into other algorithms for subsequent analysis. This configuration reduces the workload and design cost of subsequent algorithms and is beneficial to obtaining the weld quality inspection results. If the solution adopts a display method, the user will make the judgment and analysis. Since the display content has already undergone preliminary analysis and extracted information that the user can understand and recognize, it is also beneficial to obtaining the weld quality inspection results.

[0041] The principle underlying the relationship between the first grayscale image and the normal angle at each pixel location of the inspected object is as follows: Figure 2a and Figure 2b As shown. In Figure 2a The diagram illustrates an imaging device 1, a light source 2, and two objects 3 to be inspected. When the tilt of the objects changes, the light emitted from the light source 2 travels along either optical path L1 or optical path L2, resulting in varying light intensities received by the imaging device 1. This phenomenon is more pronounced on highly reflective materials. To ensure accuracy, multiple light sources 1 can be used for illumination. Figure 2b This scenario illustrates a situation involving an imaging device 1, two light sources 2, and an object under inspection 3. When the illumination angle of the light sources changes, the light emitted from the light sources 2 travels along optical path L3 or optical path L4, thereby causing a change in the light intensity received by the imaging device 1. By combining illumination from multiple angles of the light sources 2, the accuracy of the final normal angle can be guaranteed.

[0042] Further, step S20, which involves obtaining the first normal angle map of the inspected object based on the first grayscale image, includes: obtaining the first normal angle map based on a mapping model and the first grayscale image; wherein, the method for obtaining the mapping model includes the following steps: the at least two light sources sequentially illuminate the first sample to obtain a second grayscale image, wherein the number of second grayscale images corresponds to the number of light sources, and the second normal angle map of the first sample is known; and, obtaining the mapping model based on the second normal angle map and the second grayscale image. The mapping model includes a mapping relationship between grayscale values ​​and normal angles.

[0043] For example, a planar calibration plate (i.e., the first sample) can be placed in the area to be measured, the angle of the calibration plate can be adjusted, and the light source can be lit sequentially to trigger the image capture, thereby obtaining grayscale images under different lighting conditions. This yields a series of vectors {α,x,y,g1,g2…,gn}, where α is the normal angle of the corresponding measured point, x and y are the horizontal and vertical coordinates of the image pixels, and g1,g2…,gn are the pixel grayscale values ​​under different lighting angles. Based on the above vector set, machine learning can be used to establish a mapping model, that is, based on the pixel grayscale values ​​under different lighting angles and the image coordinates of the pixel, the normal angle of the corresponding area of ​​the measured surface can be inferred. In other embodiments, other methods different from machine learning can also be used to establish the mapping model, such as mathematical fitting. Once the mapping model is obtained, the first grayscale image of the object under test can be input into the mapping model to obtain a first normal angle map. In other embodiments, the planar calibration plate can also be replaced with other samples.

[0044] The normal angle mapping model corresponds to the imaging system (i.e., the position of the light source, the model of the imaging device, and the shooting position, etc.). When the imaging system changes, it should be recalibrated and learned.

[0045] Because of the special reflective properties of the hole region, the step of obtaining the first normal angle map based on the mapping model and the first grayscale image can further include: if the grayscale of a pixel in each of the first grayscale images meets the hole judgment condition, the normal angle corresponding to that pixel is set to 0°; and if the grayscale of a pixel in any of the first grayscale images does not meet the hole judgment condition, the normal angle corresponding to that pixel is obtained based on the mapping model. In one embodiment, the hole judgment condition is that the grayscale is lower than a preset grayscale value, which can be, for example, 50.

[0046] To facilitate subsequent processing, the first normal angle image is stored and displayed in grayscale. The step of obtaining the first normal angle image based on the mapping model and the first grayscale image further includes: the grayscale value of the pixel in the first normal angle image is converted and calculated according to the normal angle corresponding to the pixel, wherein 180° is converted into the maximum grayscale value, 0° is converted into the minimum grayscale value, and the other angles are converted according to the ratio.

[0047] For example, in one embodiment, the maximum grayscale value is 255 and the minimum grayscale value is 0. 180° is converted to 255 and 0° is converted to 0. The conversion formula for other angles is as follows: v = Int(x / 180*255), where v represents the converted grayscale value, x represents the normal angle value, and Int represents any rounding calculation, such as rounding to the nearest integer, rounding to the nearest whole number, or rounding up.

[0048] To improve the accuracy of the weld quality inspection method, the angle between the illumination direction of the light source and the illuminated plane of the inspected object is less than a preset angle. The preset angle can be 10°, 20°, 30°, etc. The illuminated plane should be understood as an imaginary plane obtained by mathematically fitting the surface of the inspected object; the specific fitting method can be selected according to actual needs.

[0049] The weld quality inspection method further includes the following steps: acquiring an illumination color image of the inspected object under preset conditions, wherein the pixels on the illumination color image and the first normal angle image have a corresponding relationship; and outputting and / or displaying the first normal angle image, the illumination color image, and the division result of the marked area to obtain the weld quality inspection result, wherein the weld quality inspection result includes the surface unevenness of the inspected object.

[0050] The preset lighting color map can be selected according to actual needs, such as a ring lighting color map. Specific requirements for lighting angles, light sources, and light intensity can be set according to common knowledge in the field. The information carried in the lighting color map can be cross-referenced with the first normal angle map and the division results of the marked areas to obtain more or more accurate results.

[0051] The surface roughness of the object being inspected can be determined according to... Figure 3a and Figure 3b To understand, in Figure 3a and 3b In this diagram, four light sources illuminate the object under inspection. Arrows represent the light sources and indicate their directions, while the circular area in the center represents the object under inspection. The colors (or other distinguishable characteristics) of the four light sources are C1, C2, C3, and C4, respectively. If the surface of the object under inspection is convex, its surface color is as follows: Figure 3a As shown, the area adjacent to the light source will appear the same color. If the surface of the object being inspected is concave, then the color of its surface will be as shown. Figure 3b As shown, the area opposite the light source will exhibit the same color. Therefore, the surface unevenness can be analyzed based on the color distribution pattern in the illumination color diagram.

[0052] In one embodiment, the marked region includes a horizontal region, a hole region, and a non-horizontal region. Step S30, which involves dividing the marked region based on the first normal angle map, includes: merging pixels whose normal angles belong to the horizontal determination interval to obtain at least one first sub-region; and, if the first sub-region is larger than a first preset area, the first sub-region is divided into the horizontal region. For example, when the maximum grayscale value is 255 and the minimum grayscale value is 0, the horizontal determination interval is the normal angle corresponding to the grayscale interval [120, 140]. In one embodiment, the division result of the horizontal region 14 is as follows: Figure 4a As shown, Figure 4a From left to right, the images are: the first orientation angle diagram 11, the typical region division diagram 12, and the colored illumination diagram 13. The regions shown in these three sub-diagrams correspond to the same inspected object. Figure 4b and Figure 4c The three subgraphs in the diagram can also be understood using the same approach.

[0053] Step S30, which involves dividing the marked region based on the first normal angle image, further includes: merging pixels that meet the hole determination criteria to obtain at least one second sub-region; and, if the second sub-region is larger than a second preset area, the second sub-region is classified as the hole region, or the second sub-region is expanded based on a preset rule, and the expanded second sub-region is classified as the hole region. For example, in one embodiment, the region with a grayscale of 0 in the first normal angle image is considered the hole region. It should be understood that the terms "hole determination criteria" and "hole determination conditions" appear in this specification, and the specific conditions referred to by the two conditions are different. For specific meanings, please refer to the relevant sections of this document for understanding. In one embodiment, the division result of the hole region 15 is as follows: Figure 4b As shown. In Figure 4b In the image, you can see two regions, one smaller than the other. The smaller region is the second sub-region before expansion, and the larger region is the second sub-region after expansion. The preset rules for expansion can be set according to actual needs.

[0054] Step S30, which involves dividing the marked region based on the first normal angle map, further includes: dividing the horizontal region and the hole region; clustering pixels not divided into the horizontal region and the hole region to obtain at least one third sub-region; configuring the largest area in each third sub-region as a candidate sub-region; and, if the area of ​​the candidate sub-region is greater than a third preset area and the height difference of the candidate sub-region is greater than a preset height, the candidate sub-region is classified as the non-horizontal region. The specific clustering method can be selected according to actual needs.

[0055] The elevation difference of the candidate sub-region can be calculated as follows: first, find the major axis of the third sub-region, and then calculate the elevation difference based on its length and angle. In one embodiment, the division result of the non-horizontal region 16 is as follows: Figure 4c As shown in the figure, the three line segments in the figure represent the major axes of the three non-horizontal regions 16.

[0056] The above-described steps can already help obtain high-precision weld quality inspection results. To further enhance the effectiveness of this method, this embodiment also provides a training method for training a weld quality inspection model, which includes the following steps:

[0057] S110 acquires a first grayscale image, wherein the first grayscale image is obtained by sequentially illuminating a second sample with at least two light sources, the number of first grayscale images corresponding to the number of light sources, and the light sources output parallel light. The second sample can be a selected typical sample, or the object being inspected during the operation of the weld quality inspection method can be used as the second sample.

[0058] S120 obtains the first normal angle map of the inspected object based on the first grayscale image.

[0059] S130 acquires the illumination color map of the second sample under preset conditions, wherein the illumination color map and the pixels on the first normal angle map have a corresponding relationship.

[0060] S140 divides the marked area based on the first normal angle diagram.

[0061] Furthermore, S150 trains the weld quality inspection model based on the first normal angle diagram, the lighting color diagram, and the division results of the marked area.

[0062] The weld quality inspection model trained by the aforementioned training method can be used to output weld quality inspection results based on the illumination color image of the inspected object. That is, the output of the weld quality inspection model can obtain the inspection results without the intervention of the first normal image.

[0063] The specific details of the above steps can be found in the steps described above for the weld quality inspection method.

[0064] The training method can be run independently, or relevant training images can be collected during the operation of the weld quality inspection method to improve efficiency.

[0065] In one embodiment, a transfer learning model is constructed based on a convolutional neural network, characterized by selecting different marker images for different types of surface defects. For determining the size of solder joints, the horizontal region can be used. Unsoldered areas are typically flat, with cutting marks remaining on the metal surface. Soldered areas, after being heated and melted, usually have a certain degree of surface undulation, and the cutting marks on the metal surface disappear. For concave solder joints caused by spatter, the non-horizontal region is used; in addition to the normal image, the illumination color image provides the surface texture features. For surface hole detection, the hole region is used, retaining only the normal angle and grayscale value of dark particles and their surrounding pixels, effectively avoiding interference from other image information.

[0066] Typically, surface holes and unwelded areas account for a very small proportion of the image area. Directly feeding the original image into a deep learning model makes it difficult to ensure that the features learned by the model correspond to the manual labels, which can easily lead to missed or false judgments. In this embodiment, the training images used only retain information related to the defects to be inspected, while actively removing other information. Based on human knowledge, this reduces the risk of model attention deviation and effectively improves the accuracy of model discrimination.

[0067] A specific implementation example is described below:

[0068] Surface quality inspection is performed on the laser weld seams of the stator copper busbar end face. The types of defects detected include small weld points, spatter pits, and surface holes. The camera is located above the weld point and is equipped with a large telecentric lens. Four parallel LEDs are evenly distributed around the surface being inspected, illuminating it sequentially at low angles. The camera is triggered simultaneously when the parallel lights illuminate the surface, obtaining images at different incident angles.

[0069] Step 1: A copper plate of the same material as the solder joint being tested is used as a calibration plate. It is rotated to different angles for imaging to obtain the mapping vector between pixel grayscale values ​​and normal angles. In this embodiment, the angle of the calibration copper plate varies from 0 to 180 degrees, with an interval of 5 degrees. A backpropagation (BP) neural network is used to establish a mapping model, which is stored as preset parameters in the computer.

[0070] Step 2: Position the copper solder joint to be inspected below the camera, and sequentially activate the low-angle parallel lights and trigger image capture. During solder joint inspection, in addition to parallel light illumination from different directions, the last four LED parallel lights are simultaneously illuminated and trigger the camera to capture images, obtaining a color image of the solder joint with approximately ring-shaped illumination (i.e., the illumination color image). All images are taken while keeping the camera and the solder joint under test stationary; images of the same solder joint can be directly aligned pixel-wise.

[0071] Step 3: Process all images of the solder joints, cropping them from the background image. Create a vector {x,y,g1,g2…,gn} in pixels as input to a backpropagation (BP) neural network. The network outputs the normal angle of the region corresponding to each pixel. Convert the normal angles proportionally to 1-255 to reconstruct the normal map. Mark pixels with grayscale values ​​below 50 in each image as holes, and set the grayscale value of the corresponding pixel in its normal map to 0.

[0072] Step 4: First, machine vision methods are used to traverse the normal map to find pixels with grayscale values ​​of 120-140 and 0. Pixels with grayscale values ​​in the 120-140 range correspond to a normal angle of approximately 90 degrees, which can be identified as horizontal regions. Pixels with a grayscale value of 0 correspond to surface holes with very little reflection. The area of ​​the horizontal region is calculated. If it is greater than the first preset area, the sub-plane is marked. Combined with the surrounding color image, a marked image A is obtained. Marked image A is... Figure 4a .

[0073] Step 5: Calculate the area of ​​dark particles. If the area is greater than the second preset area, mark the dark particle. Combine this with the annular illumination color map to obtain Marker Map B. Marker Map B is... Figure 4b .

[0074] Step 6: Perform cluster analysis on pixels with other grayscale values. In this embodiment, the number of clusters is set to 5. Count the area of ​​the sub-planes after cluster analysis, and select the sub-plane with the largest area in each category. If the area of ​​this sub-plane exceeds the third preset area, find the major axis of this sub-plane and calculate the drop height based on its length and angle. If the drop height is greater than a set height threshold, mark the sub-plane. Combine this with the annular illumination color map to obtain Marker Image C. Marker Image C is... Figure 4c .

[0075] Step 7: Collect images of qualified and defective solder joints. Defective solder joints include those that are too small, have spatter dents, and surface holes. Establish separate image libraries for each type of defect. For each of the three types of defects, use a ResNet50 model for transfer learning. Complete retraining of the last (top) layer and secondary training of the inner layers (first 100 layers) of the model. All learning was performed using TensorFlow 2.4.1 (CUDA 11.0) to build and train the model, with a batch size of 32, an epoch size of 10, and a learning rate of 0.00001. After continuous optimization, this embodiment achieves a 97.5% accuracy rate for identifying spatter dents, a 95.3% accuracy rate for identifying small solder joints, and a 92.6% accuracy rate for identifying surface holes.

[0076] This embodiment has the following beneficial effects:

[0077] 1. Improved the stability of the detection system, enabling the acquisition of clear images of highly reflective surfaces with high resolution, no signal loss, and good repeatability.

[0078] 2. The detection time is shortened, with the total time for a single solder joint being less than 300ms, of which the image processing and pattern recognition algorithms take less than 100ms. Compared to traditional 3D imaging detection methods, the detection time can be reduced by 50%-150%.

[0079] 3. Reduced hardware costs and procurement risks: This solution uses general industrial cameras, telecentric lenses, and LED light sources to build the imaging system. The market supply of related components is stable and there are domestically produced components. Compared with 3D cameras, it is not only cheaper, but also avoids being dependent on others for supply channels, thus reducing the procurement risks for enterprises.

[0080] In summary, this embodiment provides a weld quality inspection method and a training method. The weld quality inspection method includes the following steps: acquiring a first grayscale image, wherein the first grayscale image is obtained by sequentially illuminating the inspected object with at least two light sources, the number of first grayscale images corresponding to the number of light sources, and the light sources output parallel light; acquiring a first normal angle image of the inspected object based on the first grayscale image; dividing a marked area based on the first normal angle image; and outputting and / or displaying the first normal angle image and the division result of the marked area to obtain a weld quality inspection result. The first normal angle image provides more information, and the information it carries is more accurate, which can improve the detection accuracy of weld quality inspection, thereby solving the problem of high false positive and false negative rates when inspecting weld quality on objects with high surface reflectivity in the prior art.

[0081] The above description is only a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the present invention.

Claims

1. A method for inspecting weld quality, characterized in that, The weld quality inspection method includes the following steps: A first grayscale image is obtained, wherein the first grayscale image is obtained by sequentially illuminating the object under inspection with at least two light sources, the number of first grayscale images corresponds to the number of light sources, and the light sources output parallel light; Based on the first grayscale image, obtain the first normal angle image of the inspected object; The marked region is divided based on the first normal angle diagram; and... Output and / or display the first normal angle diagram and the division result of the marked area to obtain the weld quality inspection result; The marked region includes a horizontal region, a hole region, and a non-horizontal region. The step of dividing the marked region based on the first normal angle diagram includes: Divide the horizontal region and the hole region; For pixels that are not classified into the horizontal region and the hole region, clustering is performed to obtain at least one third sub-region; The largest area in each of the third sub-regions is configured as a candidate sub-region; and... If the area of ​​the candidate sub-region is greater than the third preset area, and the height difference of the candidate sub-region is greater than the preset height, the candidate sub-region is divided into the non-horizontal region.

2. The weld quality inspection method according to claim 1, characterized in that, The step of obtaining the first normal angle map of the inspected object based on the first grayscale image includes: obtaining the first normal angle map based on the mapping model and the first grayscale image; wherein, the method of obtaining the mapping model includes the following steps: The at least two light sources sequentially illuminate the first sample to obtain a second grayscale image, wherein the number of second grayscale images corresponds to the number of light sources, and the second normal angle image of the first sample is known; and... The mapping model is obtained based on the second normal angle map and the second grayscale map.

3. The weld quality inspection method according to claim 2, characterized in that, The step of obtaining the first normal angle image based on the mapping model and the first grayscale image includes: If the grayscale value of a pixel in each of the first grayscale images meets the hole detection criteria, the normal angle corresponding to that pixel is set to 0°; and, If the gray level of a pixel in any of the first grayscale images does not meet the hole detection criteria, the normal angle corresponding to that pixel is obtained based on the mapping model.

4. The weld quality inspection method according to claim 3, characterized in that, The first normal angle image is stored and displayed in grayscale. The step of obtaining the first normal angle image based on the mapping model and the first grayscale image further includes: The grayscale values ​​of the pixels in the first normal angle map are converted and calculated according to the normal angle corresponding to the pixel. 180° is converted to the maximum grayscale value, 0° is converted to the minimum grayscale value, and the other angles are converted proportionally.

5. The weld quality inspection method according to claim 1, characterized in that, The angle between the illumination direction of the light source and the illuminated plane of the object under inspection is less than a preset angle.

6. The weld quality inspection method according to claim 1, characterized in that, The weld quality inspection method also includes the following steps: Obtain an illumination color image of the inspected object under preset conditions, wherein the pixels in the illumination color image and the first normal angle image have a corresponding relationship; and, Output and / or display the first normal angle diagram, the illumination color diagram, and the division result of the marked area to obtain the weld quality inspection result, which includes the surface roughness of the inspected object.

7. The weld quality inspection method according to any one of claims 1 to 6, characterized in that, The marked region includes a horizontal region, and the step of dividing the marked region based on the first normal angle diagram includes: Pixels whose normal angle belongs to the horizontal determination interval are merged to obtain at least one first sub-region; and, If the first sub-region is larger than the first preset area, the first sub-region is divided into the horizontal region.

8. The weld quality inspection method according to any one of claims 1 to 6, characterized in that, The marked region includes a hole region, and the step of dividing the marked region based on the first normal angle diagram includes: Pixels that meet the hole detection criteria are merged to obtain at least one second sub-region; and, If the second sub-region is larger than the second preset area, the second sub-region is divided into the hole region; or, the second sub-region is expanded based on a preset rule, and the expanded second sub-region is divided into the hole region.

9. A training method, characterized in that, The training method for training a weld quality inspection model includes the following steps: A first grayscale image is obtained, wherein the first grayscale image is obtained by sequentially illuminating a second sample with at least two light sources, the number of first grayscale images corresponds to the number of light sources, and the light sources output parallel light; Based on the first grayscale image, obtain the first normal angle image of the inspected object; Obtain the illumination color map of the second sample under preset conditions, wherein the illumination color map and the pixels on the first normal angle map have a corresponding relationship; The marked region is divided based on the first normal angle diagram; and... The weld quality inspection model is trained based on the first normal angle map, the illumination color map, and the division result of the marked region; wherein, the weld quality inspection model is used to output the weld quality inspection result based on the illumination color map of the inspected object; The marked region includes a horizontal region, a hole region, and a non-horizontal region. The step of dividing the marked region based on the first normal angle diagram includes: Divide the horizontal region and the hole region; For pixels that are not classified into the horizontal region and the hole region, clustering is performed to obtain at least one third sub-region; The largest area in each of the third sub-regions is configured as a candidate sub-region; and... If the area of ​​the candidate sub-region is greater than the third preset area, and the height difference of the candidate sub-region is greater than the preset height, the candidate sub-region is divided into the non-horizontal region.

Images