SMT patch quality detection method and device based on machine vision and medium
Through machine vision-based detection methods, the problems of inaccurate patch image segmentation and low accuracy of appearance quality detection under rapid detection of SMT production line are solved, and higher detection accuracy and fewer error detection are achieved.
Patent Information
- Application Number
- CN202411542107.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-11
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art cannot adapt to the fast SMT production line, resulting in inaccurate patch image segmentation, reduced accuracy of appearance quality detection, and incorrect detection.
Using machine vision-based detection methods, by acquiring SMT patch images, image preprocessing, morphological reconstruction, edge extraction and dimension measurement are performed, the position area to be measured is determined, and precise measurement is performed to obtain the quality information of patch components.
It improves the accuracy of SMT patch image segmentation, improves the accuracy of patch appearance quality detection, and reduces the situation of false detection.
Smart Images
Figure CN119417804A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of quality inspection, and in particular to a method, device and medium for inspecting SMT patch quality based on machine vision. Background Art
[0002] At present, SMT technology is the abbreviation of surface mount technology, which is the most popular technology and process in the electronic assembly industry. Most of the SMT processes include: printing, furnace, mounting, cleaning, testing and other processes. SMT patches are some professional names for processing based on PCB. Every SMT product must go through the process according to the PCB data.
[0003] In one prior art, machine vision is a detection technology that uses a machine vision system to analyze images. This technology can greatly improve production efficiency and the degree of automation of production. In some dangerous working environments that are not suitable for manual operations or where artificial vision is difficult to meet the requirements, machine vision is often used to replace artificial vision. At the same time, in large-scale repetitive industrial production processes, the use of machine vision detection methods can greatly improve production efficiency and the degree of automation of production. In order to improve the accuracy of patching, it is very necessary to use online patch defect detection technology to monitor and judge the patch process in real time. Defect detection technology refers to automatic online detection of product appearance, obtaining the appearance characteristics of the product through image and signal methods, and comparing it with the set template, and determining whether there is a defect based on the confidence of the feature. However, because the degree of automation of SMT production lines is getting higher and higher, and the speed is getting faster and faster, and defect detection technology cannot adapt to rapid detection, it leads to inaccurate patch image segmentation, reduced accuracy of patch appearance quality detection after segmentation, and false detection. Summary of the invention
[0004] The present invention provides a method, device and medium for SMT patch quality detection based on machine vision, so as to improve the accuracy of SMT patch image segmentation, enhance the accuracy of SMT patch appearance quality detection and reduce false detection.
[0005] In the first aspect, in order to solve the above technical problems, the present invention provides a method, device and medium for SMT patch quality detection based on machine vision, including: Obtain the SMT patch image to be inspected collected by the machine vision module; Perform image preprocessing according to the SMT patch image to obtain image filtering information; Perform morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; According to the SMT patch edge image, the actual size of the SMT patch is obtained; Determine the location area to be measured according to the actual size of the SMT patch; Measurement is performed according to the position area to be measured to obtain quality information of the SMT patch component.
[0006] As an optional implementation manner, the step of acquiring the SMT patch image to be detected collected by the machine vision module includes: Place the machine vision module just above the geometric center of the SMT patch image to obtain a planar top view of the SMT patch; Perform image segmentation according to the plan view of the SMT patch to obtain segmented image information; Obtaining the SMT patch image information to be detected according to the segmented image information; The image segmentation is to cut the plan view of the SMT patch according to a preset segmentation area to obtain the segmented image information.
[0007] As an optional implementation manner, performing image preprocessing according to the SMT patch image to obtain image filtering information includes: According to the image of the SMT patch, color separation is performed to obtain RGB three color channel information; Calculate the grayscale value according to the RGB three color channel information to obtain the grayscale information of the image; Performing two-dimensional Gaussian filtering according to the grayscale information of the image to obtain image filtering information; The two-dimensional Gaussian filtering process includes calculating the Gaussian filtering function corresponding to the pixels on the image information: In the formula, Represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, Indicates the vertical coordinate value of the pixel coordinate in the two-dimensional coordinate system. is the Gaussian standard deviation preset by the system, express Gaussian filter function value at the coordinate point; According to the Gaussian filter function value, a convolution operation is performed with the image to obtain image filtering information; the convolution operation process is as follows: In the formula, for Image filtering information under coordinates, for The Gaussian filter function value under the coordinate, for Grayscale information at the coordinates.
[0008] As an optional implementation, performing morphological reconstruction according to the image filtering information to obtain the SMT patch edge image includes: Separating the adhesion area using a region splitting method according to the image filtering information to obtain separation information; Performing morphological reconstruction according to the separation information to obtain reconstructed image information; Obtaining an SMT patch edge image according to the reconstructed image information; The region splitting method is to select an initial seed point in the image, obtain surrounding information of the seed point according to the initial seed point, perform calculations according to the surrounding information to obtain a neighborhood average value of the seed point and a neighborhood value of the surrounding information, compare the neighborhood average value of the seed point and the neighborhood value of the surrounding information to obtain separation information; Among them, the neighboring average value of the seed point and the neighboring value of the surrounding information are compared, and the judgment is made by calculating the size of the binary image after segmentation and the original image, which is expressed by the following formula: In the formula, represents the number of pixels in the connected area in the segmented image, represents the number of pixels in the original image in the connected area, represents the projection value of the corresponding connected area, Represents a preset projection threshold matrix, which is preset by the user; wherein the connected area is an area where the gray value of the pixel differs within five; by comparing the projection values of the connected area When the difference is greater than the preset difference information, the connected area is determined as a separation area to obtain separation information.
[0009] As an optional implementation manner, performing morphological reconstruction according to the separation information to obtain reconstructed image information includes: According to the separation information, the image boundary information is obtained by gradually expanding from the inside to the outside; According to the image boundary information, performing multiple iterative dilation and erosion operations within the boundary information to obtain reconstructed image information; The dilation operation refers to the process of expanding a pixel of an image by one unit.
[0010] As an optional implementation manner, obtaining the actual size of the SMT patch according to the SMT patch edge image includes: According to the SMT patch edge image and in combination with the image resolution, the number of SMT patch edge pixels is obtained; According to the number of pixels at the edge of the SMT patch, conversion is performed to obtain the actual size of the SMT patch; The conversion is performed according to the number of pixels at the edge of the SMT patch, specifically, the actual size represented by each pixel is multiplied by the number of pixels at the edge of the SMT patch to obtain the actual area and side length of the SMT patch.
[0011] As an optional implementation manner, determining the location area to be measured according to the actual size of the SMT patch includes: According to the actual size of the SMT patch, find the point with the largest distance from the origin of the image to obtain the maximum distance information of the image; According to the actual size of the SMT patch, find the point with the shortest distance to the origin of the image to obtain the shortest distance information of the image; Comparing the maximum distance information and the shortest distance information of the image to obtain the position information to be measured; The origin of the image coordinates is the geometric center of the SMT patch image, and the end of the long side of the SMT patch image close to the bracket is the short semi-axis of the x-axis, and the end of the short side close to the bracket is the positive semi-axis of the y-axis, so as to establish a coordinate system.
[0012] As an optional implementation manner, the measuring is performed according to the position area to be measured to obtain the quality information of the SMT patch component, including: Performing linear regression fitting according to the position area to be measured to obtain linear information of the image; According to the linear information of the image, using an auxiliary straight line that perpendicularly bisects the linear information of the image, the minimum diameter and the maximum diameter of the patch component are obtained; According to the minimum diameter and the maximum diameter of the patch component, the width, the length and the inclination angle of the patch component are obtained; Obtaining the actual size of the patch component according to the width, length and inclination angle of the patch component; According to the minimum diameter and maximum diameter of the patch component, the width, length and inclination of the patch component can be expressed by the following calculation formula: In the formula, Indicates the maximum diameter of the patch component. Indicates the minimum diameter value of the patch component; In the formula, Indicates the width of the patch component. Indicates the inclination angle of the patch component, It indicates the minimum diameter of the patch component; In the formula, Indicates the length of the patch component, Indicates the width of the patch component. Indicates the inclination angle of the patch component.
[0013] In a second aspect, the present invention provides a SMT patch quality inspection device based on machine vision, characterized in that it includes: An information receiving module is used to obtain the SMT patch image to be inspected collected by the machine vision module; A preprocessing module, used for performing image preprocessing according to the SMT patch image to obtain image filtering information; An edge detection module is used to perform morphological reconstruction based on the image filtering information to obtain an SMT patch edge image; An image conversion module, used to obtain the actual size of the SMT patch according to the SMT patch edge image; A position marking module, used to determine the position area to be measured according to the actual size of the SMT patch; The information measurement module is used to perform measurement according to the position area to be measured to obtain the quality information of the SMT patch component.
[0014] In a third aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned machine vision-based SMT patch quality inspection methods.
[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method, device and medium for detecting the quality of SMT patches based on machine vision. The method is executed by a controller and includes: obtaining an SMT patch image to be detected collected by a machine vision module; performing image preprocessing according to the SMT patch image to obtain image filtering information; performing morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; obtaining the actual size of the SMT patch according to the SMT patch edge image; determining a position area to be measured according to the actual size of the SMT patch; and performing measurement according to the position area to be measured to obtain the quality information of the SMT patch component. The vision module first obtains an image of the SMT patch and preprocesses the image, including graying and Gaussian filtering, to remove noise and highlight key features; then, performing morphological reconstruction using the image filtering information to extract the edge image of the SMT patch; measuring the actual size of the SMT patch according to the obtained edge image; then, using the actual size to determine the position area to be measured; finally, performing precise measurement in the position area to obtain the quality information of the SMT patch component. This method improves the accuracy of SMT patch image segmentation, improves the accuracy of SMT patch appearance quality inspection, and reduces false detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a schematic diagram of a SMT patch quality inspection process based on machine vision provided by the first embodiment of the present invention; Figure 2 It is a schematic diagram of a SMT patch quality detection structure based on machine vision provided by the second embodiment of the present invention. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0018] Reference Figure 1 The first embodiment of the present invention provides a method for detecting the quality of SMT patches based on machine vision, comprising the following steps: S11, obtaining the SMT patch image to be inspected collected by the machine vision module; S12, performing image preprocessing according to the SMT patch image to obtain image filtering information; S13, performing morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; S14, obtaining the actual size of the SMT patch according to the SMT patch edge image; S15, determining the location area to be measured according to the actual size of the SMT patch; S16, performing measurement according to the position area to be measured to obtain quality information of the SMT patch component.
[0019] It should be noted that the acquisition of the SMT patch image to be inspected collected by the machine vision module is completed by the machine vision module, wherein the machine vision module is a key component in modern industrial automation and intelligent manufacturing. It captures and analyzes images through high-precision visual sensors to provide the necessary visual information for the automation system. This module usually includes different types of cameras, such as CCD (charge-coupled device) cameras, CMOS (complementary metal oxide semiconductor) cameras and linear array cameras, each of which has unique advantages and application scenarios. CCD cameras are known for their high resolution, high sensitivity and excellent image quality, and are particularly suitable for applications that require high-quality image capture. CMOS cameras are favored for their fast imaging capabilities, low power consumption and cost-effectiveness, and are widely used in situations that require fast response. Linear array cameras are used for continuous object detection and measurement tasks with their expertise in high-speed scanning and precise measurement. Exemplarily, in this embodiment, the machine vision module uses a CMOS camera. The CMOS camera collects the image data of the SMT patch to be inspected, and the generated digital image data contains detailed visual information of the SMT patch image to be inspected, which can be used for further image processing, analysis and decision making.
[0020] The present invention provides a method, device and medium for detecting the quality of SMT patches by machine vision. The method is executed by a controller and includes: acquiring an SMT patch image to be detected collected by a machine vision module; performing image preprocessing according to the SMT patch image to obtain image filtering information; performing morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; obtaining an actual size of the SMT patch according to the SMT patch edge image; determining a position area to be measured according to the actual size of the SMT patch; and performing measurement according to the position area to be measured to obtain quality information of the SMT patch component.
[0021] The visual module first obtains the image of the SMT patch and preprocesses the image, including grayscale and Gaussian filtering, to remove noise and highlight key features; then, the image filtering information is used for morphological reconstruction to extract the edge image of the SMT patch; based on the obtained edge image, the actual size of the SMT patch is measured; then, the actual size is used to determine the position area to be measured; finally, accurate measurement is performed in the position area to obtain the quality information of the SMT patch component. This method improves the accuracy of SMT patch image segmentation, improves the accuracy of SMT patch appearance quality detection, and reduces false detection.
[0022] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.
[0023] In step S11, the step of acquiring the SMT patch image to be detected acquired by the machine vision module includes: placing the machine vision module directly above the geometric center of the SMT patch image to obtain a planar top view of the SMT patch; Perform image segmentation according to the plan view of the SMT patch to obtain segmented image information; Obtaining the SMT patch image information to be detected according to the segmented image information; The image segmentation is to cut the plan view of the SMT patch according to a preset segmentation area to obtain the segmented image information.
[0024] It should be noted that the preset segmentation area is preset by the user, and the preset value may be an area of 20 mm×20 mm, which is not limited in the present invention.
[0025] In step S12, the image preprocessing is performed according to the SMT patch image to obtain image filtering information, including: According to the image of the SMT patch, color separation is performed to obtain RGB three color channel information; Calculate the grayscale value according to the RGB three color channel information to obtain the grayscale information of the image; Performing two-dimensional Gaussian filtering according to the grayscale information of the image to obtain image filtering information; The two-dimensional Gaussian filtering process includes calculating the Gaussian filtering function corresponding to the pixels on the image information: In the formula, Represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, Indicates the vertical coordinate value of the pixel coordinate in the two-dimensional coordinate system. is the Gaussian standard deviation preset by the system, express Gaussian filter function value at the coordinate point; According to the Gaussian filter function value, a convolution operation is performed with the image to obtain image filtering information; the convolution operation process is as follows: In the formula, for Image filtering information under coordinates, for The Gaussian filter function value under the coordinate, for Grayscale information at the coordinates.
[0026] It should be noted that the image coordinate origin of the two-dimensional coordinate system is the geometric center of the SMT patch image, and the coordinate system is established with the end of the long side of the SMT patch image close to the bracket as the short semi-axis of the x-axis, and the end of the short side close to the bracket as the positive semi-axis of the y-axis.
[0027] In step S13, morphological reconstruction is performed according to the image filtering information to obtain an SMT patch edge image, including: Separating the adhesion area using a region splitting method according to the image filtering information to obtain separation information; Performing morphological reconstruction according to the separation information to obtain reconstructed image information; Obtaining an SMT patch edge image according to the reconstructed image information; The region splitting method is to select an initial seed point in the image, obtain surrounding information of the seed point based on the initial seed point, and perform calculations based on the surrounding information to obtain a neighboring average value of the seed point and a neighboring value of the surrounding information; Among them, the neighboring average value of the seed point and the neighboring value of the surrounding information are compared to obtain the separation information; the neighboring average value of the seed point and the neighboring value of the surrounding information are compared, and the judgment is made by calculating the size of the binary image after segmentation and the original image, which is expressed by the following formula: In the formula, represents the number of pixels in the connected area in the segmented image, represents the number of pixels in the original image in the connected area, represents the projection value of the corresponding connected area, Represents a preset projection threshold matrix, which is preset by the user; wherein the connected area is an area where the gray value of the pixel differs within five; by comparing the projection values of the connected area When the difference is greater than the preset difference information, it is represented as a separation area and separation information is obtained.
[0028] It should be noted that the morphological reconstruction is performed according to the separation information to obtain the reconstructed image information, including: According to the separation information, the image boundary information is obtained by gradually expanding from the inside to the outside; According to the image boundary information, performing iterative dilation and erosion operations within the boundary information to obtain reconstructed image information; The dilation operation refers to the process of expanding a pixel of an image by one unit to its surroundings.
[0029] It should be noted that the corrosion operation is to judge the grayscale information of the pixel. If the grayscale information of the pixel point differs from the grayscale of the 3×3 square rectangular pixels centered on the pixel point by more than 20, the grayscale information is automatically adjusted to the average of the grayscale information of the surrounding square rectangles.
[0030] In step S14, obtaining the actual size of the SMT patch according to the SMT patch edge image includes: According to the SMT patch edge image and in combination with the image resolution, the number of SMT patch edge pixels is obtained; According to the number of pixels at the edge of the SMT patch, conversion is performed to obtain the actual size of the SMT patch; The conversion is performed according to the number of pixels at the edge of the SMT patch, specifically, the actual size represented by each pixel is multiplied by the number of pixels at the edge of the SMT patch to obtain the actual area and side length of the SMT patch.
[0031] In step S15, determining the location area to be measured according to the actual size of the SMT patch includes: According to the actual size of the SMT patch, find the point with the largest distance from the origin of the image to obtain the maximum distance information of the image; According to the actual size of the SMT patch, find the point with the shortest distance to the origin of the image to obtain the shortest distance information of the image; Comparing the maximum distance information and the shortest distance information of the image to obtain the position information to be measured; The origin of the image coordinates is the geometric center of the SMT patch image, and the end of the long side of the SMT patch image close to the bracket is the short semi-axis of the x-axis, and the end of the short side close to the bracket is the positive semi-axis of the y-axis, so as to establish a coordinate system.
[0032] In step S16, the measuring is performed according to the position area to be measured to obtain the quality information of the SMT patch component, including: Performing linear regression fitting according to the position area to be measured to obtain linear information of the image; According to the linear information of the image, using an auxiliary straight line that perpendicularly bisects the linear information of the image, the minimum diameter and the maximum diameter of the patch component are obtained; According to the minimum diameter and the maximum diameter of the patch component, the width, the length and the inclination angle of the patch component are obtained; Obtaining the actual size of the patch component according to the width, length and inclination angle of the patch component; According to the minimum diameter and maximum diameter of the patch component, the width, length and inclination of the patch component can be expressed by the following calculation formula: In the formula, Indicates the maximum diameter of the patch component. Indicates the minimum diameter value of the patch component; In the formula, Indicates the width of the patch component. Indicates the inclination angle of the patch component, It indicates the minimum diameter of the patch component; In the formula, Indicates the length of the patch component, Indicates the width of the patch component. Indicates the inclination angle of the patch component.
[0033] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.
[0034] In this embodiment, a SMT patch quality inspection system based on machine vision is proposed. The system consists of the following key parts: an image acquisition device, an image processing device, an edge detection and reconstruction device, a size measurement device, a positioning and calibration device, a quality assessment device, and a data processing and analysis device. Among them, the image acquisition device is used to acquire SMT patch images, and perform image preprocessing to obtain filtering information; then, the edge image of the patch is extracted through morphological reconstruction, and the actual size of the SMT patch is calculated using the edge image. The position area to be measured is determined according to the size, and the measurement is performed to obtain the quality information of the patch component. Image acquisition includes acquiring a top view of the patch through a machine vision module, segmenting it to obtain the image information to be detected; the preprocessing stage extracts image features through color separation and Gaussian filtering; morphological reconstruction uses the region splitting method to separate the adhesion area and obtain the edge image. Afterwards, the number of edge pixels is calculated and converted to the actual size, the maximum and minimum distances from the image origin are found to determine the measurement area; the width, length and inclination of the component are obtained through linear regression fitting. The corresponding device includes information reception, grayscale processing, edge detection, image conversion, position marking and information measurement modules. In addition, a computer-readable storage medium is provided, which stores a program for executing the above method. The specific working steps are as follows: Step 1: Place the machine vision module directly above the geometric center of the SMT patch image, collect the SMT patch image to be inspected, and obtain its top view. Perform image segmentation on the top view, extract the segmented image information, and obtain the SMT patch image to be inspected.
[0035] Step 2: Preprocess the collected SMT patch images. First, perform color separation to extract the three color channel information of RGB; then calculate the gray value to obtain the gray information of the image; finally, perform two-dimensional Gaussian filtering on the gray information to generate image filtering information.
[0036] Step 3: Using the image filtering information, the adhesion area in the image is separated by the region splitting method for morphological reconstruction. This process includes gradually dilating from the inside to the outside, extracting the image boundary information, and generating the edge image of the SMT patch through multiple iterative dilation and erosion operations.
[0037] Step 4: Combine the image resolution and edge image to calculate the number of edge pixels of the SMT patch. Using the actual size represented by each pixel, convert the number of edge pixels to the actual size of the patch, including area and side length.
[0038] Step 5: According to the actual size of the SMT patch, find the point with the largest and shortest distances from the image origin, obtain the maximum and minimum distance information, and determine the location area to be measured by comparison.
[0039] Step 6: Perform linear regression fitting on the determined position area to be measured to obtain the linear information of the image. Use the auxiliary straight line that perpendicularly bisects the linear information to calculate the minimum and maximum diameters of the patch component, and then obtain its width, length and inclination, and finally obtain the quality information of the SMT patch component.
[0040] Step 7: The entire detection process is completed by the corresponding device modules, including information receiving, grayscale processing, edge detection, image conversion, position marking and information measurement modules. The computer-readable storage medium stores the corresponding program to execute the above method.
[0041] Through the above steps and system design, the SMT patch quality inspection system based on machine vision of the present invention can identify the shape information of the SMT patch, obtain quality information based on the shape information of the SMT patch, and evaluate the quality, thereby improving the inspection efficiency and product quality.
[0042] In summary, the present invention provides a method, device and medium for SMT patch quality detection based on machine vision. The method is executed by a controller, including: obtaining an SMT patch image to be detected collected by a machine vision module; performing image preprocessing according to the SMT patch image to obtain image filtering information; performing morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; obtaining the actual size of the SMT patch according to the SMT patch edge image; determining the position area to be measured according to the actual size of the SMT patch; and performing measurement according to the position area to be measured to obtain the quality information of the SMT patch component. The vision module first obtains an image of the SMT patch and preprocesses the image, including graying and Gaussian filtering, to remove noise and highlight key features; then, morphological reconstruction is performed using the image filtering information to extract the edge image of the SMT patch; based on the obtained edge image, the actual size of the SMT patch is measured; then, the position area to be measured is determined using the actual size; finally, accurate measurement is performed in the position area to obtain the quality information of the SMT patch component. This method improves the accuracy of SMT patch image segmentation, improves the accuracy of SMT patch appearance quality inspection, and reduces false detection.
[0043] Reference Figure 2 The second embodiment of the present invention provides a SMT patch quality detection device based on machine vision, comprising: An information receiving module is used to obtain the SMT patch image to be inspected collected by the machine vision module; A preprocessing module, used for performing image preprocessing according to the SMT patch image to obtain image filtering information; An edge detection module is used to perform morphological reconstruction based on the image filtering information to obtain an SMT patch edge image; An image conversion module, used to obtain the actual size of the SMT patch according to the SMT patch edge image; A position marking module, used to determine the position area to be measured according to the actual size of the SMT patch; The information measurement module is used to perform measurement according to the position area to be measured to obtain the quality information of the SMT patch component.
[0044] In one implementation, the information receiving module includes: Place the machine vision module directly above the geometric center of the SMT patch image to obtain a plan view of the SMT patch; perform image segmentation based on the plan view of the SMT patch to obtain segmented image information; obtain image information of the SMT patch to be detected based on the segmented image information; wherein the image segmentation is to cut the plan view of the SMT patch according to a preset segmentation area to obtain the segmented image information.
[0045] In one implementation, the grayscale processing module includes: According to the image of the SMT patch, color separation is performed to obtain RGB three color channel information; Calculate the grayscale value according to the RGB three color channel information to obtain the grayscale information of the image; Performing two-dimensional Gaussian filtering according to the grayscale information of the image to obtain image filtering information; The two-dimensional Gaussian filtering process includes calculating the Gaussian filtering function corresponding to the pixels on the image information: In the formula, Represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, Indicates the vertical coordinate value of the pixel coordinate in the two-dimensional coordinate system. is the Gaussian standard deviation preset by the system, express Gaussian filter function value at the coordinate point; According to the Gaussian filter function value, a convolution operation is performed with the image to obtain image filtering information; the convolution operation process is as follows: In the formula, for Image filtering information under coordinates, for The Gaussian filter function value under the coordinate, for Grayscale information at the coordinates.
[0046] In one implementation, the edge detection module includes: Separating the adhesion area using a region splitting method according to the image filtering information to obtain separation information; Performing morphological reconstruction according to the separation information to obtain reconstructed image information; Obtaining an SMT patch edge image according to the reconstructed image information; The region splitting method is to select an initial seed point in the image, obtain surrounding information of the seed point according to the initial seed point, perform calculations according to the surrounding information to obtain a neighborhood average value of the seed point and a neighborhood value of the surrounding information, compare the neighborhood average value of the seed point and the neighborhood value of the surrounding information to obtain separation information; Among them, the neighboring average value of the seed point and the neighboring value of the surrounding information are compared, and the judgment is made by calculating the size of the binary image after segmentation and the original image, which is expressed by the following formula: In the formula, represents the number of pixels in the connected area in the segmented image, represents the number of pixels in the original image in the connected area, represents the projection value of the corresponding connected area, Represents a preset projection threshold matrix, which is preset by the user; wherein the connected area is an area where the gray value of the pixel differs within five; by comparing the projection values of the connected area When the difference is greater than the preset difference information, it is represented as a separation area, and separation information is obtained. Wherein, performing morphological reconstruction according to the separation information to obtain reconstructed image information includes: According to the separation information, the image boundary information is obtained by gradually expanding from the inside to the outside; According to the image boundary information, performing iterative dilation and erosion operations within the boundary information to obtain reconstructed image information; The dilation operation refers to the process of expanding a pixel of an image by one unit to its surroundings.
[0047] In one implementation, the image conversion module includes: According to the SMT patch edge image and in combination with the image resolution, the number of SMT patch edge pixels is obtained; According to the number of pixels at the edge of the SMT patch, conversion is performed to obtain the actual size of the SMT patch; The conversion is performed according to the number of pixels at the edge of the SMT patch, specifically, the actual size represented by each pixel is multiplied by the number of pixels at the edge of the SMT patch to obtain the actual area and side length of the SMT patch.
[0048] In one embodiment, the position marking module includes: According to the actual size of the SMT patch, find the point with the largest distance from the origin of the image to obtain the maximum distance information of the image; According to the actual size of the SMT patch, find the point with the shortest distance to the origin of the image to obtain the shortest distance information of the image; Comparing the maximum distance information and the shortest distance information of the image to obtain the position information to be measured; The origin of the image coordinates is the geometric center of the SMT patch image, and the end of the long side of the SMT patch image close to the bracket is the short semi-axis of the x-axis, and the end of the short side close to the bracket is the positive semi-axis of the y-axis, so as to establish a coordinate system.
[0049] In one implementation, the information measurement module includes: Performing linear regression fitting according to the position area to be measured to obtain linear information of the image; According to the linear information of the image, using an auxiliary straight line that perpendicularly bisects the linear information of the image, the minimum diameter and the maximum diameter of the patch component are obtained; According to the minimum diameter and the maximum diameter of the patch component, the width, the length and the inclination angle of the patch component are obtained; Obtaining the actual size of the patch component according to the width, length and inclination angle of the patch component; According to the minimum diameter and maximum diameter of the patch component, the width, length and inclination of the patch component can be expressed by the following calculation formula: In the formula, Indicates the maximum diameter of the patch component. Indicates the minimum diameter value of the patch component; In the formula, Indicates the width of the patch component. Indicates the inclination angle of the patch component, It indicates the minimum diameter of the patch component; In the formula, Indicates the length of the patch component, Indicates the width of the patch component. Indicates the inclination angle of the patch component.
[0050] It should be noted that the SMT patch quality inspection device based on machine vision provided in an embodiment of the present invention is used to execute all the process steps of the SMT patch inspection method based on machine vision in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and thus will not be elaborated on.
[0051] In summary, the present invention provides a method, device and medium for SMT patch quality detection based on machine vision. The method is executed by a controller, including: obtaining an SMT patch image to be detected collected by a machine vision module; performing image preprocessing according to the SMT patch image to obtain image filtering information; performing morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; obtaining the actual size of the SMT patch according to the SMT patch edge image; determining the position area to be measured according to the actual size of the SMT patch; and performing measurement according to the position area to be measured to obtain the quality information of the SMT patch component. The vision module first obtains an image of the SMT patch and preprocesses the image, including graying and Gaussian filtering, to remove noise and highlight key features; then, morphological reconstruction is performed using the image filtering information to extract the edge image of the SMT patch; based on the obtained edge image, the actual size of the SMT patch is measured; then, the position area to be measured is determined using the actual size; finally, accurate measurement is performed in the position area to obtain the quality information of the SMT patch component. This method improves the accuracy of SMT patch image segmentation, improves the accuracy of SMT patch appearance quality inspection, and reduces false detection.
[0052] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an image segmentation program. When the processor executes the computer program, the steps in the above-mentioned various machine vision-based SMT patch quality detection method embodiments are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are realized, such as the information receiving module.
[0053] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.
[0054] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.
[0055] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.
[0056] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0057] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0058] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.
[0059] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.
Claims
1. A SMT patch quality detection method based on machine vision, characterized in that: include: Obtain the SMT patch image to be inspected collected by the machine vision module; Perform image preprocessing according to the SMT patch image to obtain image filtering information; Perform morphological reconstruction according to the image filtering information to obtain an SMT patch edge image; According to the SMT patch edge image, the actual size of the SMT patch is obtained; Determine the location area to be measured according to the actual size of the SMT patch; Measurement is performed according to the position area to be measured to obtain quality information of the SMT patch component.
2. The SMT patch quality detection method based on machine vision according to claim 1 is characterized in that: The step of obtaining the SMT patch image to be detected collected by the machine vision module includes: Place the machine vision module just above the geometric center of the SMT patch image to obtain a planar top view of the SMT patch; Perform image segmentation according to the plan view of the SMT patch to obtain segmented image information; Obtaining the SMT patch image information to be detected according to the segmented image information; The image segmentation is to cut the plan view of the SMT patch according to a preset segmentation area to obtain the segmented image information.
3. The SMT patch quality detection method based on machine vision according to claim 1 is characterized in that: The step of performing image preprocessing according to the SMT patch image to obtain image filtering information includes: According to the image of the SMT patch, color separation is performed to obtain RGB three color channel information; Calculate the grayscale value according to the RGB three color channel information to obtain the grayscale information of the image; Performing two-dimensional Gaussian filtering according to the grayscale information of the image to obtain image filtering information; The two-dimensional Gaussian filtering process includes calculating the Gaussian filtering function corresponding to the pixels on the image information: In the formula, Represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, Indicates the vertical coordinate value of the pixel coordinate in the two-dimensional coordinate system. is the Gaussian standard deviation preset by the system, express Gaussian filter function value at the coordinate point; According to the Gaussian filter function value, a convolution operation is performed with the image to obtain image filtering information; the convolution operation process is as follows: In the formula, for Image filtering information under coordinates, for The Gaussian filter function value under the coordinate, for Grayscale information at the coordinates.
4. The SMT patch quality detection method based on machine vision according to claim 1 is characterized in that: The step of performing morphological reconstruction according to the image filtering information to obtain an SMT patch edge image includes: Separating the adhesion area using a region splitting method according to the image filtering information to obtain separation information; Performing morphological reconstruction according to the separation information to obtain reconstructed image information; Obtaining an SMT patch edge image according to the reconstructed image information; The region splitting method is to select an initial seed point in the image, obtain surrounding information of the seed point according to the initial seed point, perform calculations according to the surrounding information to obtain a neighborhood average value of the seed point and a neighborhood value of the surrounding information, compare the neighborhood average value of the seed point and the neighborhood value of the surrounding information to obtain separation information; Among them, the neighboring average value of the seed point and the neighboring value of the surrounding information are compared, and the judgment is made by calculating the size of the binary image after segmentation and the original image, which is expressed by the following formula: In the formula, represents the number of pixels in the connected area in the segmented image, represents the number of pixels in the original image in the connected area, represents the projection value of the corresponding connected area, Represents a preset projection threshold matrix, which is preset by the user; wherein the connected area is an area where the gray value of the pixel differs within five; by comparing the projection values of the connected area When the difference is greater than the preset difference information, the connected area is determined as a separation area to obtain separation information.
5. The SMT patch quality detection method based on machine vision according to claim 4 is characterized in that: The step of performing morphological reconstruction according to the separation information to obtain reconstructed image information comprises: According to the separation information, the image boundary information is obtained by gradually expanding from the inside to the outside; According to the image boundary information, performing iterative dilation and erosion operations within the boundary information to obtain reconstructed image information; Among them, dilation refers to the process of expanding a pixel of an image by one unit to its surrounding area.
6. The SMT patch quality detection method based on machine vision according to claim 1 is characterized in that: The step of obtaining the actual size of the SMT patch according to the SMT patch edge image includes: According to the SMT patch edge image and in combination with the image resolution, the number of SMT patch edge pixels is obtained; According to the number of pixels at the edge of the SMT patch, conversion is performed to obtain the actual size of the SMT patch; The conversion is performed according to the number of pixels at the edge of the SMT patch, specifically, the actual size represented by each pixel is multiplied by the number of pixels at the edge of the SMT patch to obtain the actual area and side length of the SMT patch.
7. The SMT patch quality detection method based on machine vision according to claim 1 is characterized in that: Determining the location area to be measured according to the actual size of the SMT patch includes: According to the actual size of the SMT patch, find the point with the largest distance from the origin of the image to obtain the maximum distance information of the image; According to the actual size of the SMT patch, find the point with the shortest distance to the origin of the image to obtain the shortest distance information of the image; Comparing the maximum distance information and the shortest distance information of the image to obtain the position information to be measured; The origin of the image coordinates is the geometric center of the SMT patch image, and the end of the long side of the SMT patch image close to the bracket is the short semi-axis of the x-axis, and the end of the short side close to the bracket is the positive semi-axis of the y-axis, so as to establish a coordinate system.
8. The SMT patch quality detection method based on machine vision according to claim 1 is characterized in that: The step of measuring the position area to be measured to obtain quality information of the SMT patch component includes: Performing linear regression fitting according to the position area to be measured to obtain linear information of the image; According to the linear information of the image, using an auxiliary straight line that perpendicularly bisects the linear information of the image, the minimum diameter and the maximum diameter of the patch component are obtained; According to the minimum diameter and the maximum diameter of the patch component, the width, the length and the inclination angle of the patch component are obtained; Obtaining the actual size of the patch component according to the width, length and inclination angle of the patch component; According to the minimum diameter and maximum diameter of the patch component, the width, length and inclination of the patch component can be expressed by the following calculation formula: In the formula, Indicates the maximum diameter of the patch component. Indicates the minimum diameter value of the patch component; In the formula, Indicates the width of the patch component. Indicates the inclination angle of the patch component, It indicates the minimum diameter of the patch component; In the formula, Indicates the length of the patch component, Indicates the width of the patch component. Indicates the inclination angle of the patch component.
9. A SMT patch quality inspection device based on machine vision, characterized in that: include: An information receiving module is used to obtain the SMT patch image to be inspected collected by the machine vision module; A preprocessing module, used for performing image preprocessing according to the SMT patch image to obtain image filtering information; An edge detection module is used to perform morphological reconstruction based on the image filtering information to obtain an SMT patch edge image; An image conversion module, used to obtain the actual size of the SMT patch according to the SMT patch edge image; A position marking module, used to determine the position area to be measured according to the actual size of the SMT patch; The information measurement module is used to perform measurement according to the position area to be measured to obtain the quality information of the SMT patch component.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the SMT patch quality detection method based on machine vision as described in any one of claims 1 to 8.