Micro connector production quality inspection method and system based on computer vision

Abstract

The present invention relates to the technical field of micro-connector image recognition, and specifically to a micro-connector production quality inspection method and system based on computer vision. Images are classified according to preset defect types and defect areas are extracted. An optional reference image is selected, and the degree of deformation and the degree of contour disorder are calculated through morphological features, and the degree of defect offset is calculated in combination with the distance distribution to form geometric features. The data distribution of the image in the geometric features is analyzed to obtain the protrusion degree, and then the geometric authenticity is calculated. According to the gradient distribution of the pins in the reference image, the light attenuation angle and intensity are extracted as illumination features. The optical authenticity is calculated by the differences in geometric features, illumination features and spectral correlation coefficients of images of the same type, and an authenticity evaluation model is constructed to generate a final sample for micro-connector production quality inspection. The present invention can generate defect image samples that conform to real physical conditions, so that the samples can be used to accurately detect the production quality of micro-connectors.

Description

Technical Field

[0001] The present invention relates to the technical field of micro-connector image recognition, and in particular to a method and system for detecting the production quality of micro-connectors based on computer vision. Background Art

[0002] A micro connector is a miniaturized electrical connection component used to connect two circuits or devices to transmit signals, power, or data. This type of connector typically features a small size and a high-density interface design, making it suitable for space-constrained applications such as portable devices, medical equipment, consumer electronics, and industrial automation systems. During the production phase, micro connector pin defects typically account for over 70%, and these defects significantly impact connector performance, potentially affecting key performance factors such as the device's electrical performance, mechanical strength, and reliability. As a smaller industrial product, any tiny defect in the pins will affect the performance of the micro connector, and such defects cannot be detected by manual inspection. Therefore, using computer vision to inspect the production quality of micro connectors is a key step in improving overall product quality.

[0003] When using computer vision to inspect the production quality of micro-connectors, machine learning methods are often combined to build defect detection models. However, building models often requires a large amount of data as training samples, including normal samples and defective samples. Currently, the process production level of micro-connectors is relatively high, especially after the introduction of automated processes, where the defect rate can be reduced to below 1%. This makes sample collection difficult, and the problem of a far insufficient number of defective samples often arises. To expand defect samples, existing technical methods use generative adversarial networks (GANs) to generate more realistic data samples. However, micro-connector production scenarios require very high precision, and samples generated by GANs may exhibit manufacturing defect patterns that do not conform to real physical conditions, making it impossible to use the generated samples to accurately inspect the production quality of micro-connectors. Summary of the Invention

[0004] In order to solve the technical problem that the micro-connector production scenario has high precision requirements, the samples generated by GAN may have manufacturing defect patterns that do not conform to the real physical situation, and thus the generated samples cannot be used to accurately detect the production quality of micro-connectors. The purpose of the present invention is to provide a micro-connector production quality detection method and system based on computer vision. The technical scheme adopted is as follows: A micro-connector production quality detection method based on computer vision, the method comprises: obtaining a connector defect image of the micro-connector after processing by a GAN model; classifying all connector defect images into a preset number of defect types; obtaining the defect area in each connector defect image; selecting any one connector defect image as a reference image; obtaining the degree of deformation and the degree of contour disorder of the reference image based on the area characteristics and contour characteristics of the defect area in the reference image; obtaining the degree of defect offset of the reference image based on the distance distribution characteristics between the defect area and the nearest connector pin; and The degree of disorder and the degree of defect offset are used as geometric features in all connector defect images; the prominence of each geometric feature is obtained based on the data changes of all connector defect images on each geometric feature; the geometric authenticity of the reference image is obtained based on the data value of each geometric feature and the prominence of each geometric feature of the reference image; the illumination attenuation angle and illumination attenuation intensity of the reference image are obtained based on the gradient distribution characteristics of all pin pixels in the reference image; the illumination attenuation angle and illumination attenuation intensity are used as illumination features in all connector defect images; the optical authenticity of the reference image is obtained based on the geometric feature differences, illumination feature differences and spectral correlation coefficients between the reference image and the connector defect images of the same defect type; the true evaluation model of the reference image is obtained based on the geometric authenticity and optical authenticity of the reference image; the GAN model is corrected based on the true evaluation model to obtain the final defect image sample; the production quality of the micro connector is inspected based on the final defect image sample.

[0005] Furthermore, the method for obtaining the degree of deformation and the degree of contour disorder of the reference image includes: obtaining the minimum enclosing rectangle of the defect area in the reference image; taking the product of the area of ​​the minimum enclosing rectangle and the ratio between the length and width of the minimum enclosing rectangle as the degree of deformation of the reference image; calculating the contour curvature entropy of the defect area according to the curvature entropy formula as the degree of contour disorder of the reference image.

[0006] Furthermore, the method for obtaining the defect offset degree includes: obtaining the defect offset degree according to a defect offset degree calculation formula, and the defect offset degree calculation formula is as follows: Where, Indicates the degree of defect offset from the reference image; Indicates a preset first number of pins that are closest to a centroid pixel point of the defective area in the reference image; Indicates the distance between the centroid pixel of the defect area and the nearest Euclidean distance between pins; represents the minimum function; Represents the maximum function.

[0007] Furthermore, the method for obtaining the protrusion degree includes: selecting the maximum and minimum values ​​of each geometric feature in the reference image and other connector defect images of the same defect type, thereby calculating the range value of each geometric feature as the relevant range value of each geometric feature; calculating the range value of each geometric feature in all connector defect images of each defect type as the defect type range of each geometric feature, and calculating the mean of the defect type ranges corresponding to all defect types of each geometric feature as the overall range mean of each geometric feature; performing negative correlation normalization processing on the ratio between the relevant range value of each geometric feature and the overall range mean to obtain the protrusion degree of each geometric feature.

[0008] Furthermore, the method for obtaining the geometrical authenticity includes: selecting any geometric feature as a reference feature; In principle, the data values ​​corresponding to the reference features of all connector defect images are analyzed to obtain the authenticity index of the data values ​​corresponding to the reference features in the reference images; the geometric authenticity is obtained according to the geometric authenticity calculation formula, which is as follows: Where, Indicates the geometric authenticity of the reference image; Indicates the number of geometric features; Indicates the The prominence of each geometric feature; Indicates the reference image The authenticity indicator of the data value corresponding to each geometric feature; Represents an exponential function with a natural constant as its base.

[0009] Furthermore, the method for obtaining the light attenuation angle and light attenuation intensity includes: selecting the pin closest to the center pixel point of the reference image as the center pin, and taking the pixel point corresponding to the vertex of the center pin as the reference pixel point; establishing a Cartesian coordinate system with the lower left corner of the reference image as the origin; establishing a reference area with the reference pixel point as the center; the number of pins contained in the reference area is not less than a preset first number, and the reference area does not include defective pins; obtaining the minimum gradient value within a preset neighborhood of each pixel point in the reference area, and the direction angle between the direction of the minimum gradient value and the positive axis of the x-axis; calculating the average of the absolute values ​​of the minimum gradient values ​​of each pixel point in the reference area within the preset neighborhood as the light attenuation intensity of the reference image; calculating the average of the direction angles between the direction of the minimum gradient value of each pixel point in the reference area within the preset neighborhood and the positive axis of the x-axis as the light attenuation angle of the reference image.

[0010] Furthermore, the method for obtaining the optical realism includes: combining data values ​​corresponding to all illumination features and all geometric features of the reference image into a feature vector of the reference image; and obtaining the optical realism according to an optical realism calculation formula, which is as follows: Where, Indicates the degree of optical authenticity of the reference image; Indicates the number of connector defect images with the same defect type as the reference image; represents the reference image; Indicates the serial number of the connector defect image with the same defect type as the reference image; Indicates that the reference image and the defect type are the same Spectral correlation coefficient between the connector defect images; The feature vector representing the reference image; Indicates the same defect type as the reference image. Feature vector of the connector defect image; The feature vector of the reference image is the same as the defect type. The cosine similarity between the feature vectors of the connector defect images.

[0011] Furthermore, the method for obtaining the true evaluation model includes: calculating the mean value of the data value discreteness of each geometric feature of the reference image and other connector defect images of the same defect type as the geometric discreteness of the reference image; calculating the mean value of the data value discreteness of each illumination feature of the reference image and other connector defect images of the same defect type as the optical discreteness of the reference image; obtaining the model geometric weight and the model optical weight based on the geometric discreteness and the optical discreteness of the reference image; obtaining the true evaluation model of the reference image based on the geometric authenticity, model geometric weight, optical authenticity and model optical weight of the reference image.

[0012] A micro-connector production quality inspection system based on computer vision, the system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned micro-connector production quality inspection method based on computer vision are implemented.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for detecting the production quality of micro-connectors based on computer vision.

[0014] The present invention has the following beneficial effects: the present invention obtains a connector defect image of a micro connector after being processed by a GAN model, so as to facilitate physical constraints on the connector defect image; since in actual production situations, it is necessary to ensure that the generated defect morphology conforms to the geometric constraints of the real physical world, such as the bending radius of the pin cannot be too small, and the crack morphology must conform to the stress law, etc., the geometric features of the defect area in the image are analyzed, and the geometric features include: degree of deformation, degree of contour disorder, and degree of defect offset; since different types of defects are contained in different connector defect images, the prominent geometric features are different for images of different defect types, so the protrusion degree of each geometric feature is obtained according to the data changes of all connector defect images on each geometric feature, and then the geometric authenticity of the reference image is obtained; optical characteristics are also a key characteristic for evaluating image authenticity, and the generated defect image must conform to objective physical laws, otherwise it will cause optical distortion of the image. Because each real defect image may be captured with varying lighting and angles, the optical characteristics of the image are analyzed, including the angle and intensity of light attenuation. The optical authenticity of the reference image is determined based on the geometric and illumination feature differences and spectral correlation coefficients between the reference image and a connector defect image of the same defect type. A true evaluation model for the reference image is then obtained. The GAN model is modified based on the true evaluation model to obtain a final defect image sample. The final defect image sample is then used to perform production quality inspection on the micro-connector. The present invention can generate defect image samples that conform to real physical conditions, enabling accurate inspection of micro-connector production quality using the samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A flowchart of a method for detecting the production quality of micro-connectors based on computer vision according to an embodiment of the present invention; Figure 2 A block diagram of a computer vision-based micro-connector production quality inspection system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0017] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effectiveness of the computer vision-based micro-connector production quality inspection method and system proposed by the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] The specific scheme of the micro-connector production quality inspection method and system based on computer vision provided by the present invention is described in detail below with reference to the accompanying drawings.

[0020] See also Figure 1 , which shows a micro-connector production quality inspection method based on computer vision provided by an embodiment of the present invention, the method comprising: step S1: obtaining a connector defect image of the micro-connector after being processed by a GAN model; and classifying all connector defect images into a preset number of defect types.

[0021] The embodiment of the present invention is mainly used in the scenario of generating defect images of micro connectors that can be used for actual factory production. In the embodiment of the present invention, a generative adversarial network (GAN) is used to generate relatively realistic micro connector samples, and the generated samples may have manufacturing defect patterns that do not conform to the actual physical conditions. Therefore, the connector defect image of the micro connector after being processed by the GAN model is first obtained, and the connector defect image is physically constrained. The specific method is shown in the subsequent steps.

[0022] In this embodiment of the present invention, defect types are categorized into bent pins, scratches, and nicks, with the preset number set to 3. Because different connector defect images may contain multiple defects, and the defect conditions within each connector defect image vary, in this embodiment of the present invention, the defect type with the highest number of defects in each connector defect image is used as the defect type for that connector defect image. It should be noted that the defect type and preset number can be customized and are not limited here.

[0023] Step S2: Obtain the defect area in each connector defect image; select any connector defect image as a reference image; obtain the deformation degree and contour disorder degree of the reference image based on the area characteristics and contour characteristics of the defect area in the reference image; obtain the defect offset degree of the reference image based on the distance distribution characteristics between the defect area and the nearest connector pin; use the deformation degree, contour disorder degree and defect offset degree as geometric features in all connector defect images; obtain the protrusion degree of each geometric feature based on the data changes of all connector defect images on each geometric feature; obtain the geometric truth degree of the reference image based on the data value of each geometric feature of the reference image and the protrusion degree of each geometric feature.

[0024] Connector defect images generated by conventional GAN ​​models may contain manufacturing defect patterns that do not conform to physical rules, such as discontinuous scratches, overly regular gaps, and unnatural wear. Therefore, it is necessary to verify the authenticity of the defects in the connector defect images. In actual production situations, it is necessary to ensure that the generated defect morphology conforms to the geometric constraints of the real physical world, such as the bending radius of the pin cannot be too small and the crack morphology must conform to stress laws. Therefore, in this embodiment of the present invention, the geometric features of the defect area in the image are analyzed.

[0025] First, all defective regions in each connector defect image are obtained. A defective region in any connector defect image is selected as the reference region in the reference image. Possible geometric deformations in the defective region include deformed components, wear on the connector surface, or pin displacement.

[0026] Preferably, in one embodiment of the present invention, the method for obtaining the degree of deformation and the degree of contour disorder of a reference image includes: obtaining the minimum enclosing rectangle of the reference area; and taking the product of the area of ​​the minimum enclosing rectangle and the ratio between the length and width of the minimum enclosing rectangle as the degree of deformation of the reference area.

[0027] The contour curvature entropy of the reference area is calculated according to the curvature entropy formula as the degree of disorder of the contour of the reference area. The specific formula is as follows: Where, Indicates the degree of disorder of the contour of the reference area; The number of partitions of the principal curvature histogram representing the contour of the defect area; The first histogram of the principal curvature representing the contour of the defect area partitions; is the principal curvature histogram of the defect area contour The distribution probability of each partition; represents a logarithmic function with base 10. It should be noted that the above formula is a technical means well known to those skilled in the art and will not be described in detail here.

[0028] Preferably, in one embodiment of the present invention, the method for obtaining the defect offset degree includes: obtaining the defect offset degree according to a defect offset degree calculation formula, and the defect offset degree calculation formula is as follows: Where, Indicates the degree of defect deviation in the reference area; Indicates a preset first number of pins that are closest to the centroid pixel of the reference area; Indicates the distance between the centroid pixel of the reference area and the nearest Euclidean distance between pins; represents the minimum function; Represents the maximum function.

[0029] In one embodiment of the present invention, the preset first number is set to 4. It should be noted that the preset first number can be set arbitrarily and is not limited here.

[0030] In the defect offset calculation formula, the ratio of the minimum distance to the maximum distance between the center of mass of the defect area and the four adjacent pin vertices The smaller it is, the greater the offset of the reference area is. The larger it is, the greater the degree of defect deviation in the reference area.

[0031] Since different connector defect images contain different types of defects, the protruding geometric features are different for images of different defect types. Therefore, in an embodiment of the present invention, the protrusion degree of each geometric feature is obtained based on the data changes of all connector defect images on each geometric feature.

[0032] Preferably, in one embodiment of the present invention, the method for obtaining the protrusion degree includes: selecting the maximum and minimum values ​​of each geometric feature in the reference image and other connector defect images of the same defect type, thereby calculating the range value of each geometric feature as the relevant range value of each geometric feature; calculating the range value of each geometric feature in all connector defect images of each defect type as the defect type range of each geometric feature, and calculating the mean of the defect type ranges corresponding to all defect types of each geometric feature as the overall range mean of each geometric feature.

[0033] The ratio between the correlation range value of each geometric feature and the overall range mean is negatively normalized to obtain the prominence of each geometric feature in the reference image. The calculation formula is as follows: Where, Indicates the The prominence of each geometric feature; Indicates the The relative range of geometric features; Indicates the The overall mean range of the geometric features.

[0034] In the protrusion degree calculation formula, the ratio between the relevant range value of each geometric feature and the overall range mean is used to express the fluctuation range of the defect type of the reference image on each geometric feature. The smaller it is, the smaller the fluctuation range is, which means that the geometric feature can more effectively affect the detection result of the connector defect image of this defect type, and the more prominent the geometric feature is.

[0035] Preferably, in one embodiment of the present invention, the method for obtaining the geometrical authenticity comprises: selecting any one geometric feature as a reference feature; The data values ​​corresponding to the reference features of all connector defect images are analyzed in principle to obtain the authenticity index of the data values ​​corresponding to the reference features in the reference images. The principle is a technical means well known to those skilled in the art. Here is an example: Since the data values ​​corresponding to the reference features of all connector defect images satisfy the normal distribution, if the corresponding data values ​​of the reference features of the reference image fall within In the case of , the authenticity index is 0.682. If the corresponding data value of the reference feature of the reference image falls within In addition, If the corresponding data value of the reference feature of the reference image falls within In addition, If the value is within , the authenticity index is 0.146.

[0036] The geometric true degree is obtained according to the geometric true degree calculation formula. The geometric true degree calculation formula is as follows: Where, Indicates the geometric authenticity of the reference image; Indicates the number of geometric features; Indicates the The prominence of each geometric feature; Indicates the reference image The authenticity indicator of the data value corresponding to each geometric feature; Represents an exponential function with a natural constant as its base.

[0037] In the geometric truth calculation formula, the first The larger the authenticity index of the data value corresponding to the geometric feature is, the The higher the authenticity of the first geometric feature in the reference image, the higher the The greater the prominence of the geometric features, the The higher the prominence of the first geometric feature in the reference image, the higher the The larger the authenticity weight of a geometric feature, the same analysis is performed on each geometric feature, and finally the geometric authenticity of the reference image is obtained.

[0038] Optical properties are also a key factor in assessing image authenticity. Generated defect images must conform to objective physical laws; otherwise, optical distortion will occur. Because each real defect image may be captured with varying lighting and angles, this embodiment of the present invention selects real images that are generally similar to the generated image and uses a similarity metric between their optical appearances to determine the optical authenticity of the generated image.

[0039] Since the pins are made of metal material, they have a better reflection effect on light, which is reflected in the image as a higher pixel performance. Therefore, in an embodiment of the present invention, the light attenuation angle and light attenuation intensity of the reference image are obtained based on the gradient distribution characteristics of all pin pixels in the reference image.

[0040] Preferably, in one embodiment of the present invention, the method for obtaining the light attenuation angle and the light attenuation intensity includes: selecting the pin closest to the center pixel point of the reference image as the center pin, and taking the pixel point corresponding to the vertex of the center pin as the reference pixel point.

[0041] A Cartesian coordinate system is established with the lower left corner of the reference image as the origin, the horizontal side as the x-axis, and the vertical side as the y-axis; a reference area is established with the reference pixel point as the center; the number of pins contained in the reference area is not less than a preset first number, and defective pins are not included in the reference area; the present invention provides a method for obtaining a reference area, which specifically includes: establishing a first neighborhood with the reference pixel point as the center, and expanding the neighborhood outward along a preset direction, and the neighborhood remains square after each expansion (for example, initially 1×1, 3×3 after one layer of expansion, and 5×5 after two layers), until the total number of pin vertices in the neighborhood reaches more than 1 / 4 of the total number of pin vertices in the image, and in order to prevent defective pins from affecting the acquisition of optical features, if the neighborhood expansion process includes pin vertices in the defective area, the neighborhood continues to expand in the opposite direction of the original expansion direction, and finally determines the position of the neighborhood, and uses the neighborhood at this time as the reference area; here, the preset direction is the positive axis direction of the x-axis.

[0042] Obtain the minimum gradient value within the preset neighborhood of each pixel point in the reference area, and the direction angle between the direction of the minimum gradient value and the positive x-axis; calculate the average of the absolute values ​​of the minimum gradient values ​​of each pixel point in the reference area within the preset neighborhood as the light attenuation intensity of the reference image; calculate the average of the direction angles between the direction of the minimum gradient value of each pixel point in the preset neighborhood and the positive x-axis as the light attenuation angle of the reference image.

[0043] The illumination attenuation angle and illumination attenuation intensity are used as illumination features in all connector defect images. The optical authenticity of the reference image is obtained based on the geometric feature differences, illumination feature differences, and spectral correlation coefficients between the reference image and connector defect images of the same defect type.

[0044] Preferably, in one embodiment of the present invention, the method for obtaining the optical realism includes: combining data values ​​corresponding to all illumination features and all geometric features of the reference image into a feature vector of the reference image; the vector representation is as follows: Where, The feature vector representing the reference image; Indicates the illumination attenuation angle of the reference image; Represents the illumination attenuation intensity of the reference image; Indicates the degree of deformation of the reference image; Indicates the degree of contour disorder of the reference image; Indicates the degree of defect deviation from the reference image.

[0045] The optical realism degree is obtained according to the optical realism degree calculation formula, which is as follows: Where, Indicates the degree of optical authenticity of the reference image; Indicates the number of connector defect images with the same defect type as the reference image; represents the reference image; Indicates the serial number of the connector defect image with the same defect type as the reference image; Indicates that the reference image and the defect type are the same The spectral correlation coefficient between the connector defect images can be directly obtained by the existing technology and will not be described in detail here; The feature vector representing the reference image; Indicates the same defect type as the reference image. Feature vector of the connector defect image; The feature vector of the reference image is the same as the defect type. The cosine similarity between the feature vectors of the connector defect images.

[0046] In the optical true degree calculation formula, the reference image and the defect type are the same. Spectral correlation coefficient between the connector defect images The larger the value, the more similar the grayscale frequency components of the two images are, and the closer the illumination performance between the two images is. In this case, the optical authenticity of the reference image is greater. The eigenvector of the reference image is the same as the first image with the same defect type. The greater the cosine similarity between the feature vectors of the connector defect images, the closer the reference image is to the second connector defect image with the same defect type. The closer the two connector defect images are in terms of geometry, the more credible the comparison of the illumination representations of the two objects will be, and the greater the optical authenticity of the reference image.

[0047] Step S4: Obtain a true evaluation model of the reference image based on the geometric and optical realities of the reference image; modify the GAN model based on the true evaluation model to obtain a final defect image sample; and perform production quality inspection on the micro connector based on the final defect image sample.

[0048] In this embodiment of the present invention, a physical evaluation layer is embedded at the output of the GAN model's generator to assess whether the defect images produced by the generator have good physical authenticity. This physical evaluation layer uses geometric and optical realism to construct a realistic evaluation model.

[0049] Preferably, in one embodiment of the present invention, the method for obtaining a true evaluation model includes: calculating the mean value of the discrete degree of data values ​​of the reference image and other connector defect images of the same defect type in each geometric feature as the geometric discrete degree of the reference image; calculating the mean value of the discrete degree of data values ​​of the reference image and other connector defect images of the same defect type in each illumination feature as the optical discrete degree of the reference image; obtaining the model geometric weight and the model optical weight according to the geometric discrete degree and the optical discrete degree of the reference image; in one embodiment of the present invention, the sum of the geometric discrete degree and the optical discrete degree is used as the first sum; the ratio between the geometric discrete degree and the first sum is used as the model geometric weight; and the ratio between the optical discrete degree and the first sum is used as the model optical weight.

[0050] According to the geometric authenticity, model geometric weight, optical authenticity and model optical weight of the reference image, the true evaluation model of the reference image is obtained. The calculation formula of the true evaluation model is as follows: Where, represents the true evaluation model value of the reference image; Represents the model geometric weight of the reference image; Indicates the geometric authenticity of the reference image; represents the model optical weight of the reference image; Indicates the degree of optical authenticity of the reference image; Represents the normalization function.

[0051] In one embodiment of the present invention, the GAN model is modified according to the real evaluation model to obtain the final defect image sample, including: modifying the loss function of the GAN generator according to the real evaluation model value of each connector defect image to obtain a modified generator loss function, as shown in the following formula: Where, Indicates the The corrected generator loss function for the connector defect image; Indicates the The original generator loss function for the connector defect image; Indicates the The true evaluation model value of the connector defect image.

[0052] In the above formula, since is the normalized value, and we use Adjust the original generator loss function to increase it; The true evaluation model value of the connector defect image The larger the value, the closer the image is to the real situation, the smaller the required correction strength is, and the smaller the change in the generator loss function after correction is; on the contrary, The smaller the true evaluation model value of a connector defect image, the worse the authenticity of the image, the greater the required correction strength, and the greater the change in the corrected generator loss function.

[0053] The final defect image sample is generated by the modified GAN generator, and the image quality in the final defect image sample is higher.

[0054] The defect detection model is re-optimized and rebuilt using the final defect image samples. The optimized model has a high level of detection accuracy and fits well with the factory's actual production process, enabling it to efficiently complete targeted inspection tasks for micro connectors.

[0055] In summary, the connector defect images of the micro connector after being processed by the GAN model are obtained; all the connector defect images are divided into a preset number of defect types; all the defect areas in each connector defect image are obtained; one connector defect image is selected as a reference image; the deformation degree and contour disorder degree of the reference image are obtained according to the area characteristics and contour characteristics of the defect area in the reference image; the defect offset degree of the reference image is obtained according to the distance distribution characteristics between the defect area and the nearest connector pin; the deformation degree, contour disorder degree and defect offset degree are used as geometric features in all connector defect images; the prominence degree of each geometric feature is obtained according to the data changes of all connector defect images on each geometric feature; the prominence degree of each geometric feature is obtained according to the reference image on each geometric feature. The geometric realism of the reference image is obtained by combining the data value of the feature and the prominence of each geometric feature; the illumination attenuation angle and illumination attenuation intensity of the reference image are obtained according to the gradient distribution characteristics of all pin pixels in the reference image; the illumination attenuation angle and illumination attenuation intensity are used as illumination features in all connector defect images; the optical realism of the reference image is obtained according to the geometric feature differences, illumination feature differences and spectral correlation coefficients between the reference image and connector defect images of the same defect type; the real evaluation model of the reference image is obtained according to the geometric realism and optical realism of the reference image; the GAN model is corrected according to the real evaluation model to obtain the final defect image sample; and the production quality of the micro connector is inspected based on the final defect image sample.

[0056] A second objective of one embodiment of the present invention is to provide a computer vision-based micro-connector production quality inspection system. The system comprises a memory, a processor, and a computer program. The memory is configured to store the computer program, and the processor is configured to execute the computer program. When executed by the processor, the computer program implements the method described in steps S1-S4. Specifically, the system comprises: an image acquisition module 101 configured to acquire defective images of micro-connectors after processing using a GAN model; and to classify all defective connector images into a predetermined number of defect types.

[0057] The geometric feature analysis module 102 is used to obtain the defect area in each connector defect image; select any connector defect image as a reference image; obtain the deformation degree and contour disorder degree of the reference image based on the area characteristics and contour characteristics of the defect area in the reference image; obtain the defect offset degree of the reference image based on the distance distribution characteristics between the defect area and the nearest connector pin; use the deformation degree, contour disorder degree and defect offset degree as geometric features in all connector defect images; obtain the protrusion degree of each geometric feature based on the data changes of all connector defect images on each geometric feature; obtain the geometric truth degree of the reference image based on the data value of each geometric feature of the reference image and the protrusion degree of each geometric feature.

[0058] The optical feature analysis module 103 is used to obtain the light attenuation angle and light attenuation intensity of the reference image based on the gradient distribution characteristics of all pin pixels in the reference image; use the light attenuation angle and light attenuation intensity as the light features in all connector defect images; and obtain the optical authenticity of the reference image based on the geometric feature differences, light feature differences, and spectral correlation coefficients between the reference image and connector defect images of the same defect type.

[0059] The detection module 104 is used to obtain a true evaluation model of the reference image based on the geometric and optical realism of the reference image; modify the GAN model based on the true evaluation model to obtain a final defect image sample; and perform production quality inspection on the micro connector based on the final defect image sample.

[0060] The third object of an embodiment of the present invention is to provide a computer medium, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in steps S1-S4 is implemented when the processor executes the computer program.

[0061] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0062] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A method for detecting the production quality of micro connectors based on computer vision, characterized in that: The method includes: obtaining a connector defect image of a micro connector after being processed by a GAN model; classifying all connector defect images into a preset number of defect types; obtaining a defect area in each connector defect image; selecting any connector defect image as a reference image; obtaining a deformation degree of the reference image based on the area characteristics and contour characteristics of the defect area in the reference image; calculating the contour curvature entropy of the defect area according to a curvature entropy formula as the contour disorder degree of the reference image; obtaining a defect offset degree of the reference image according to the distance distribution characteristics between the defect area and the nearest connector pin; using the deformation degree, the contour disorder degree and the defect offset degree as geometric features in all connector defect images; obtaining a protrusion degree of each geometric feature based on the data change of all connector defect images on each geometric feature; obtaining a geometric truth degree of the reference image based on the data value of each geometric feature of the reference image and the protrusion degree of each geometric feature; obtaining a light attenuation angle and a light attenuation intensity of the reference image based on the gradient distribution characteristics of all pin pixels in the reference image; using the light attenuation angle and the light attenuation intensity as the light attenuation in all connector defect images Features; according to the geometric feature differences, illumination feature differences and spectral correlation coefficients between the reference image and the connector defect image of the same defect type, the optical authenticity of the reference image is obtained; according to the geometric authenticity and optical authenticity of the reference image, a true evaluation model of the reference image is obtained; according to the true evaluation model, the GAN model is corrected to obtain a final defect image sample; according to the final defect image sample, the production quality inspection of the micro connector is performed; the method for obtaining the protrusion degree includes: in the reference image and other connector defect images of the same defect type, the maximum and minimum values ​​of each geometric feature are selected to calculate the range value of each geometric feature as the relevant range value of each geometric feature; in all connector defect images of each defect type, the range value of each geometric feature is calculated as the defect type range of each geometric feature, and the mean of the defect type ranges of each geometric feature corresponding to all defect types is calculated as the overall range mean of each geometric feature; the ratio between the relevant range value of each geometric feature and the overall range mean is negatively correlated and normalized to obtain the protrusion degree of each geometric feature; the geometric authenticity calculation formula is as follows: Where, Indicates the geometric authenticity of the reference image; Indicates the number of geometric features; Indicates the The prominence of each geometric feature; Indicates the reference image The authenticity indicator of the data value corresponding to each geometric feature; represents an exponential function with a natural constant as the base; the optical reality calculation formula is as follows: Where, Indicates the degree of optical authenticity of the reference image; Indicates the number of connector defect images with the same defect type as the reference image; represents the reference image; Indicates the serial number of the connector defect image with the same defect type as the reference image; Indicates that the reference image and the defect type are the same Spectral correlation coefficient between the connector defect images; The feature vector representing the reference image; Indicates the same defect type as the reference image. Feature vector of the connector defect image; The feature vector of the reference image is the same as the defect type. The cosine similarity between the feature vectors of the connector defect images.

2. A method for detecting the production quality of micro connectors based on computer vision according to claim 1, characterized in that: The method for obtaining the deformation degree of the reference image includes: obtaining the minimum bounding rectangle of the defect area in the reference image; and taking the product of the area of ​​the minimum bounding rectangle and the ratio between the length and width of the minimum bounding rectangle as the deformation degree of the reference image.

3. The method for detecting the production quality of micro connectors based on computer vision according to claim 1, characterized in that: The method for obtaining the defect offset degree includes: obtaining the defect offset degree according to a defect offset degree calculation formula, and the defect offset degree calculation formula is as follows: Where, Indicates the degree of defect offset from the reference image; Indicates a preset first number of pins that are closest to a centroid pixel point of the defective area in the reference image; Indicates the distance between the centroid pixel of the defect area and the nearest Euclidean distance between pins; represents the minimum function; Represents the maximum function.

4. The method for detecting the production quality of micro connectors based on computer vision according to claim 1, wherein: The method for obtaining the authenticity index includes: selecting any geometric feature as a reference feature; In principle, the data values ​​corresponding to the reference features of all connector defect images are analyzed to obtain the authenticity index of the data values ​​corresponding to the reference features in the reference images.

5. The method for detecting the production quality of micro connectors based on computer vision according to claim 1, characterized in that: The method for obtaining the light attenuation angle and light attenuation intensity includes: selecting the pin closest to the center pixel point of the reference image as the center pin, and the pixel point corresponding to the vertex of the center pin as the reference pixel point; establishing a Cartesian coordinate system with the lower left corner of the reference image as the origin; establishing a reference area with the reference pixel point as the center; the number of pins contained in the reference area is not less than a preset first number, and the reference area does not include defective pins; obtaining the minimum gradient value within a preset neighborhood of each pixel point in the reference area, and the direction angle between the direction with the minimum gradient value and the positive axis of the x-axis; calculating the average of the absolute values ​​of the minimum gradient values ​​of each pixel point in the reference area within the preset neighborhood as the light attenuation intensity of the reference image; calculating the average of the direction angles between the direction with the minimum gradient value of each pixel point in the reference area within the preset neighborhood and the positive axis of the x-axis as the light attenuation angle of the reference image.

6. The method for detecting the production quality of micro connectors based on computer vision according to claim 1, characterized in that: The method for obtaining the feature vector includes: combining data values ​​corresponding to all illumination features and all geometric features of the reference image into a feature vector of the reference image.

7. The method for detecting the production quality of micro connectors based on computer vision according to claim 1, characterized in that: The method for obtaining the true evaluation model includes: calculating the mean value of the data value discreteness of each geometric feature of the reference image and other connector defect images of the same defect type as the geometric discreteness of the reference image; calculating the mean value of the data value discreteness of each illumination feature of the reference image and other connector defect images of the same defect type as the optical discreteness of the reference image; obtaining the model geometric weight and the model optical weight according to the geometric discreteness and the optical discreteness of the reference image; and obtaining the true evaluation model of the reference image according to the geometric authenticity, the model geometric weight, the optical authenticity and the model optical weight of the reference image.

8. A micro-connector production quality inspection system based on computer vision, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the micro-connector production quality inspection method based on computer vision as described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a method for detecting the production quality of micro connectors based on computer vision as described in any one of claims 1 to 7 are implemented.

Images