Welding gun mainboard defect detection method and system based on image processing

Abstract

The present invention relates to the field of image data processing technology, and in particular to a welding gun mainboard defect detection method and system based on image processing. The method comprises: extracting the grayscale image of the welding gun mainboard surface image by semantic segmentation to obtain a mainboard area image; dividing the mainboard area image into a plurality of image blocks according to a preset size; recording any image block as a target image block, calculating the grayscale difference degree of the target image block; calculating the potential degree of false solder joints of the target image block; determining the defective block according to the size of the potential degree of false solder joints; using the maximum entropy method to perform threshold segmentation processing on each defective block in turn, obtaining the false solder joint area of ​​each defective block, and completing welding gun mainboard defect detection based on image processing. The present invention effectively identifies subtle defects on the surface of the welding gun mainboard by analyzing the grayscale difference degree of the image blocks, combining edge detection and chain code recognition methods, and improving detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a welding gun mainboard defect detection method and system based on image processing. Background Art

[0002] With the development of industries such as electronic products and communication equipment, welding technology plays an important role in various industries. As the core component of welding equipment, the production quality of welding gun motherboard directly affects the welding effect and product quality. In the traditional production of welding gun motherboard, manual inspection has problems such as high cost, low efficiency and difficulty in finding small defects, which leads to defective products entering the market and affecting product quality and brand image. With the continuous advancement of image processing technology, the popularization of automated inspection systems can not only improve production efficiency, but also reduce labor costs and improve production consistency and stability. In the welding process, due to improper temperature control or too short welding time, a kind of cold soldering defect will be generated. Cold soldering defect means that the surface of the solder joint seems to have been welded, but the actual internal connection is poor, which will cause the electrical contact between the pin and the pad of the electronic component to be loose or even completely disconnected, thereby affecting the welding quality and product performance. An existing method for detecting potential cold soldering defects on the surface of welding gun motherboard is the maximum entropy method, which can automatically calculate the optimal threshold, is suitable for complex images, and has strong adaptability.

[0003] The patent application document with publication number CN115760830A provides a fabric defect detection method, including: using adaptive median filtering to denoise the fabric image; using a fuzzy domain image enhancement method to enhance the image; using the Sobel edge difference operator to calculate the edge gradient amplitude of the image in the horizontal, vertical, 45° and 135° directions; using linear interpolation method to perform non-maximum suppression according to the edge gradient amplitude; using iteration method and maximum entropy method to segment and fuse the image, and calculate the high and low thresholds of the compressed image; after suppressing isolated pixels according to the high and low thresholds, extracting the edge of the defective image.

[0004] However, there may be flux residues around the solder joints on the surface of the welding gun motherboard. Although these flux residues will not affect the reliability of the electrical connection, they may give the solder joint surface an appearance similar to a cold solder joint, thereby affecting the accuracy of potential cold solder joint detection; at the same time, the basic idea of ​​the maximum entropy method in image segmentation is to determine a suitable threshold by maximizing the entropy value of the image grayscale histogram. Usually, the maximum entropy method uses a global threshold, that is, the grayscale histogram of the entire image is analyzed and a global threshold is selected for image segmentation. However, using a fixed global threshold will result in inaccurate segmentation results when segmenting each block area on the image, resulting in the loss of important information. Summary of the invention

[0005] In order to solve the problem that the accuracy of cold solder joint detection is reduced due to flux residue, and at the same time, when using the maximum entropy method, the segmentation results of each block of the image using a fixed global threshold are inaccurate, thereby causing the loss of important information, the present invention provides a welding gun mainboard defect detection method and system based on image processing.

[0006] In a first aspect, the present invention provides a welding gun mainboard defect detection method based on image processing, which adopts the following technical solution:

[0007] The welding gun mainboard defect detection method based on image processing includes: extracting the grayscale image of the welding gun mainboard surface image by using semantic segmentation to obtain the mainboard area image; dividing the mainboard area image into a plurality of image blocks according to a preset size, recording any image block as a target image block, and calculating the grayscale difference degree of the target image block based on the grayscale value of each grayscale pixel point in the target image block, the grayscale mean of the target image block, and the variance of the number of grayscale pixels corresponding to each grayscale level in the grayscale histogram corresponding to the target image block; using edge detection to obtain all edges and edge pixels of each image block, and The chain code is used to identify the direction of the edge pixel point, and the direction number value of the edge pixel point is obtained; based on the grayscale difference degree, the number of edge directions of the target image block, and the difference between the direction number values ​​of all two adjacent edge pixels on each edge of the target image block, the potential degree of cold solder joint of the target image block is calculated, and the edge direction number is the number of types of all direction number values ​​in the target image block; according to the size of the potential cold solder joint degree, the defective block is determined; the maximum entropy method is used to perform threshold segmentation processing on each defective block in turn, to obtain the cold solder joint area of ​​each defective block, and complete the welding gun motherboard defect detection based on image processing.

[0008] The beneficial effects are as follows: by analyzing the grayscale difference degree of image blocks, combined with edge detection and chain code recognition methods, subtle defects on the surface of the welding gun motherboard, such as cold solder joints, can be effectively identified, thereby improving the accuracy of cold solder joint detection; using grayscale value, mean and histogram variance analysis, the influence of flux residue on the detection results can be effectively reduced, thereby avoiding misjudgment and missed judgment caused by residual substances; using the maximum entropy method to perform adaptive threshold segmentation on each defective block, the segmentation threshold can be dynamically adjusted according to the characteristics of different image areas, avoiding information loss caused by using a fixed global threshold, and ensuring more accurate defective area identification; combined with a comprehensive analysis of grayscale differences, edge directions and potential cold solder joints, the quality of the welding gun motherboard can be evaluated from multiple dimensions, thereby improving the comprehensiveness and accuracy of defect detection.

[0009] Furthermore, the semantic segmentation adopts the DeepLab model.

[0010] The beneficial effects are: the deep neural network model used for semantic segmentation can accurately segment the surface of the welding gun motherboard and accurately identify defective areas; it has stronger adaptability to complex backgrounds and ensures stable performance under complex conditions.

[0011] Furthermore, the grayscale difference degree satisfies the following relationship:

[0012] ; In the formula, For the The grayscale difference of each image block, For the The number of grayscale pixels in an image block, For the The first image block The gray value of a gray pixel, For the The grayscale mean of each image block, For the The variance of the number of grayscale pixels corresponding to each grayscale level in the grayscale histogram of each image block is is the absolute value symbol.

[0013] The beneficial effects are: by calculating the difference between the grayscale value and the grayscale mean of each pixel in the image block, the local grayscale changes can be accurately captured, thereby improving the accuracy of image analysis; the changes in grayscale distribution are weighted in combination with the grayscale histogram variance, making areas with larger grayscale changes more prominent and increasing sensitivity to subtle differences; and by calculating the absolute value difference, noise and real changes can be effectively distinguished, thereby reducing false detections and improving robustness.

[0014] Furthermore, the edge detection uses the Canny operator.

[0015] Furthermore, the chain code adopts an 8-neighborhood chain code.

[0016] Furthermore, the potential degree of cold solder joints satisfies the following relationship:

[0017] ; In the formula, For the The potential degree of solder joint failure in each image block, For the The grayscale difference of each image block, For the The number of types of direction number values ​​of edge pixels of each image block, For the The number of edges in each image block, For the The first image block The number of edge pixels on the edge of the strip, and Respectively The first image block The first and The direction number value of the edge pixel point, is the normalization function, is the absolute value symbol.

[0018] The beneficial effects are: combining grayscale differences, edge numbers and edge direction changes can effectively identify potential cold solder joint areas, thereby improving the accuracy of cold solder joint detection; comprehensively considering grayscale differences and edge features makes the detection process more comprehensive and can accurately capture subtle changes in the image; using normalization processing to reduce the scale differences of different image blocks makes the detection results more stable and consistent.

[0019] Further, the determining of the defective block according to the potential degree of solder joint failure includes: in response to the potential degree of solder joint failure being greater than a preset defect threshold, identifying the image block as a defective block.

[0020] The beneficial effects are: by setting a defect threshold, when the potential degree of poor solder joint exceeds this value, it is identified as a defective block, effectively avoiding misjudgment and missed judgment; it can automatically divide the image blocks into defective and non-defective areas, improve detection efficiency, and reduce manual intervention.

[0021] In a second aspect, the present invention provides a welding gun mainboard defect detection system based on image processing, which adopts the following technical solution:

[0022] The welding gun mainboard defect detection system based on image processing comprises: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the welding gun mainboard defect detection method based on image processing is implemented.

[0023] By adopting the above technical solution, the above-mentioned welding gun mainboard defect detection method based on image processing is generated into a computer program and stored in a memory so as to be loaded and executed by a processor, thereby making a terminal device based on the memory and the processor for easy use.

[0024] The present invention has the following technical effects:

[0025] Since the mainboard area image of the welding gun may contain flux residue areas similar to potential cold solder joint defects, a single global threshold cannot accurately adapt to the segmentation requirements of different areas, resulting in reduced segmentation accuracy; the mainboard area image of the welding gun is divided into blocks and the grayscale features are independently analyzed in each block image to evaluate the degree of potential cold solder joints. For each block with a large degree of potential cold solder joints, the maximum entropy method is used for image segmentation separately, which can more accurately capture the subtle changes in these areas, avoid the appearance of cold solder joints caused by factors such as flux residue, and the inaccurate segmentation problem caused by the use of a global threshold, thereby improving the segmentation effect and the sensitivity of defect detection and reducing the risk of losing important information. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding numbers are the same or corresponding parts.

[0027] Figure 1 It is a flow chart of a method for detecting defects in a welding gun mainboard based on image processing in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0029] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0030] The core purpose of the present invention is to use the maximum entropy method to perform threshold segmentation processing on the potential cold soldering defects on the surface of the welding gun mainboard. Before segmentation, the welding gun mainboard image will be divided into blocks, and based on the analysis of the grayscale features in each image block, each image block that needs to be processed separately using the maximum entropy method is screened out.

[0031] The embodiment of the present invention discloses a welding gun mainboard defect detection method based on image processing, referring to Figure 1, comprising steps S1 to S6:

[0032] S1: Semantic segmentation is used to extract the grayscale image of the welding gun motherboard surface image to obtain the motherboard area image.

[0033] Specifically, the semantic segmentation adopts the DeepLab model.

[0034] The surface of the welding gun mainboard is photographed vertically by a high-definition camera to obtain the surface image of the welding gun mainboard. The surface image of the welding gun mainboard is then grayscaled to obtain a grayscale map of the surface image of the welding gun mainboard. In order to facilitate the analysis of the grayscale features of the surface of the welding gun mainboard in subsequent steps, the Deep Convolutional Neural Network for Semantic Image Segmentation version 3 (DeepLabV3) model is used to extract the welding gun mainboard area in the surface image of the welding gun mainboard.

[0035] S2: Divide the mainboard area image into a number of image blocks according to preset sizes.

[0036] It should be noted that, by performing image block processing on the image of the welding gun mainboard area, in order to facilitate the analysis in subsequent steps, the specific blocking rule for the image of the welding gun mainboard area is: the image of the welding gun mainboard area is equally divided into several image blocks, and for the grayscale pixel points located in the edge area of ​​the welding gun mainboard and the surrounding grayscale pixel points are not enough to construct an image block, the grayscale pixel point is merged into the image block with the closest Euclidean distance to the grayscale pixel point for analysis.

[0037] Implementers can set the size of the image blocks according to specific implementation conditions, for example, 7×7.

[0038] S3: Record any image block as a target image block, and calculate the grayscale difference degree of the target image block.

[0039] It should be noted that the grayscale difference degree of each image block is obtained by analyzing the grayscale value performance of all grayscale pixels in each image block; when analyzing this indicator, due to the obvious grayscale value changes in the potential cold solder joint area on the surface of the welding gun mainboard, the more the grayscale value of the grayscale pixel in each image block deviates from the grayscale mean value in the block, and the greater the difference in the number of grayscale pixels corresponding to each gray level in the grayscale histogram, it can be explained that the more discrete the grayscale value distribution in this image block, the greater the grayscale difference, and the greater the possibility of potential cold solder joint defects in this image block.

[0040] The grayscale difference degree of the target image block is calculated based on the grayscale value of each grayscale pixel in the target image block, the grayscale mean of the target image block, and the variance of the number of grayscale pixels corresponding to each grayscale level in the grayscale histogram corresponding to the target image block.

[0041] Specifically, the grayscale difference degree satisfies the following relationship:

[0042] ;

[0043] In the formula, For the The grayscale difference of each image block, For the The number of grayscale pixels in an image block, For the The first image block The gray value of a gray pixel, For the The grayscale mean of each image block, For the The variance of the number of grayscale pixels corresponding to each grayscale level in the grayscale histogram of each image block is is the absolute value symbol.

[0044] in, Indicates The grayscale value of all grayscale pixels in an image block is the offset of the grayscale mean value of the image block. The larger the value, the larger the offset, which can explain the The more discrete the gray value distribution in an image block is, the greater the gray value difference is, and the greater the possibility of potential solder joint defects in this image block is. The larger the The greater the difference in the number of pixels corresponding to each gray level in the grayscale histogram of the image blocks, the greater the difference in the number of pixels corresponding to each gray level in the image blocks. The greater the offset of the grayscale value of all grayscale pixels in the image block compared to the grayscale mean value of the image block, the higher the credibility, which means that The more discrete the gray value distribution within an image block, the greater the gray value difference, which can further explain the The greater the possibility that there is a potential cold soldering defect in each image block.

[0045] S4: Calculate the potential degree of solder joint failure of the target image block.

[0046] It should be noted that in order to distinguish between potential cold solder joints and residual flux areas in the surface image of the welding gun motherboard, the grayscale change characteristics in each image block are further analyzed, and then the potential cold solder joint degree in each image block is calculated in combination with the grayscale difference degree analysis in each image block; before analyzing this indicator, solder joints with potential cold solder joints usually have irregular or incomplete edges, and may have missing parts, resulting in rough and incomplete edges of solder joints, or obvious fractures on the edges; and residual flux usually does not change the edge shape of the solder joints, and there are no obvious fractures or abrupt changes. Therefore, when analyzing the potential cold solder joint degree in each image block, the more different direction numbers obtained using the chain code in an image block, the greater the difference in direction numbers between adjacent pixel points on all edges in an image block, which means that the more chaotic the grayscale changes in this image block, the greater the possibility of potential cold solder joint defects in this block, and the greater the corresponding potential cold solder joint degree.

[0047] Edge detection is used to obtain all edges and edge pixels of each image block, and the direction of the edge pixel points is identified by using the chain code to obtain the direction number value of the edge pixel points; based on the grayscale difference degree, the number of edge directions of the target image block, and the difference between the direction number values ​​of all two adjacent edge pixels on each edge of the target image block, the potential degree of solder joint failure of the target image block is calculated, and the edge direction number is the number of types of all direction number values ​​in the target image block.

[0048] Specifically, the edge detection uses the Canny operator.

[0049] Specifically, the chain code adopts an 8-neighborhood chain code.

[0050] Specifically, the potential degree of cold solder joints satisfies the following relationship:

[0051] ;

[0052] In the formula, For the The potential degree of solder joint failure in each image block, For the The grayscale difference of each image block, For the The number of types of direction number values ​​of edge pixels of each image block, For the The number of edges in each image block, For the The first image block The number of edge pixels on the edge of the strip, and Respectively The first image block The first and The direction number value of the edge pixel point, is the normalization function, is the absolute value symbol.

[0053] in, The bigger, the The greater the grayscale difference between the image blocks, the greater the possibility of potential cold soldering defects in this image block, and the greater the corresponding potential cold soldering degree; The larger the value is, the more chaotic the grayscale changes in this image block are, the greater the possibility of potential cold soldering defects in this image block, and the greater the corresponding potential cold soldering degree; for image blocks without potential cold soldering defects, It will be 0, and the corresponding potential degree of solder joint failure will also be 0. The image block does not need to be processed separately using the maximum entropy method in the subsequent process. The larger it is, the greater the difference in direction numbers between two adjacent pixels on all edges in an image block is, which can confirm that the more chaotic the grayscale changes in this image block are, the higher the credibility is, and further indicates that the potential degree of solder joint failure in this image block is greater.

[0054] S5: Determine defect blocks according to the potential degree of solder joint failure.

[0055] Specifically, the step of determining defect blocks according to the potential degree of solder joint failure includes:

[0056] In response to the potential degree of poor solder joint being greater than a preset defect threshold, the image block is identified as a defective block.

[0057] Implementers can set the defect threshold according to specific implementation conditions, for example, 0.5.

[0058] S6: Use the maximum entropy method to perform threshold segmentation processing on each defect block in turn to obtain the cold soldering area of ​​each defect block, and complete the welding gun mainboard defect detection based on image processing.

[0059] For image blocks that do not need to be processed by the maximum entropy method alone, since the potential degree of cold solder joints in these areas is low, there may be no potential cold solder joint defects. At the same time, there will be no obvious grayscale changes in these areas, and the distribution of grayscale histograms is usually smoother, so these areas do not need to be processed using the maximum entropy method; and for all defect blocks that need to be processed by the maximum entropy method alone, the maximum entropy method is used for threshold segmentation processing, and the corresponding segmentation thresholds are obtained to obtain the cold solder joint areas. The obtained cold solder joint areas are the potential cold solder joint defects on the surface of the welding gun motherboard. The specific maximum entropy method threshold segmentation processing is an existing well-known technology and will not be repeated here.

[0060] An embodiment of the present invention also discloses a welding gun mainboard defect detection system based on image processing, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a welding gun mainboard defect detection method based on image processing according to the present invention is implemented.

[0061] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

[0062] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, or any other medium that can be used to store the required information and can be accessed by an application, module, or both. Any such computer storage medium may be part of the device or accessible or connectable to the device.

[0063] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0064] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A welding gun mainboard defect detection method based on image processing, characterized in that: include: Semantic segmentation is used to extract the grayscale image of the welding gun mainboard surface image to obtain the mainboard area image; The mainboard area image is equally divided into a number of image blocks according to a preset size, and any image block is recorded as a target image block. The grayscale difference degree of the target image block is calculated based on the grayscale value of each grayscale pixel point in the target image block, the grayscale mean value of the target image block, and the variance of the number of grayscale pixels corresponding to each grayscale level in the grayscale histogram corresponding to the target image block; All edges and edge pixels of each image block are obtained by edge detection, and the direction of the edge pixel is identified by chain code to obtain the direction number value of the edge pixel; based on the grayscale difference degree, the number of edge directions of the target image block, and the difference between the direction number values ​​of all two adjacent edge pixels on each edge of the target image block, the potential degree of solder joint failure of the target image block is calculated, and the edge direction number is the number of types of all direction number values ​​in the target image block; The potential degree of cold solder joints satisfies the following relationship: ； In the formula, For the The potential degree of solder joint failure in each image block, For the The grayscale difference of each image block, For the The number of types of direction number values ​​of edge pixels of each image block, For the The number of edges in each image block, For the The first image block The number of edge pixels on the edge of the strip, and Respectively The first image block The first and The direction number value of the edge pixel point, is the normalization function, is the absolute value symbol; Determine defect blocks according to the potential degree of solder joint failure; The maximum entropy method is used to perform threshold segmentation processing on each defect block in turn to obtain the cold soldering area of ​​each defect block, and complete the welding gun motherboard defect detection based on image processing.

2. The welding gun mainboard defect detection method based on image processing according to claim 1 is characterized in that: The semantic segmentation adopts the DeepLab model.

3. The welding gun mainboard defect detection method based on image processing according to claim 1 is characterized in that: The grayscale difference degree satisfies the following relationship: ； In the formula, For the The grayscale difference of each image block, For the The number of grayscale pixels in an image block, For the The first image block The gray value of a gray pixel, For the The grayscale mean of each image block, For the The variance of the number of grayscale pixels corresponding to each grayscale level in the grayscale histogram of each image block is is the absolute value symbol.

4. The welding gun mainboard defect detection method based on image processing according to claim 1 is characterized in that: The edge detection uses the Canny operator.

5. The welding gun mainboard defect detection method based on image processing according to claim 1 is characterized in that: The chain code adopts an 8-neighborhood chain code.

6. The welding gun mainboard defect detection method based on image processing according to claim 1 is characterized in that: Determining defect blocks according to the potential degree of solder joint failure includes: In response to the potential degree of poor solder joint being greater than a preset defect threshold, the image block is identified as a defective block.

7. The welding gun mainboard defect detection system based on image processing is characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the welding gun mainboard defect detection method based on image processing according to any one of claims 1 to 6 is implemented.

Images