Connector foreign matter automatic detection method, device, medium and system based on image recognition

Abstract

The invention discloses a connector foreign matter automatic detection method, device, medium and system based on image recognition, and belongs to the field of electronic product assembly, and the method comprises the steps: carrying out the pixel-level segmentation of the edge of a connector before detection, and completing the foreign matter judgment process through the local registration and similarity comparison learning of an image. According to the invention, the dependence on a large amount of connector training data is eliminated, the influence of external interference of the connector and camera shake on detection is reduced, reliable assembly of the connector in an electronic product is realized, and finally stable and effective transmission of signals is realized.

Description

Technical Field

[0001] The present invention relates to the field of electronic product assembly, and more specifically, to an automatic detection method, device, medium and system for connector foreign matter based on image recognition. Background Art

[0002] Traditional anomaly detection methods are generally based on careful image preprocessing and are sensitive to parameters. Deep learning-based anomaly detection methods require manual labeling of a large number of anomaly samples and then end-to-end training of the mainstream target detection framework as a detector. However, in reality, there are dozens of connectors for a single large electronic device alone. Collecting huge amounts of training data for each connector and deploying different models for different categories is time-consuming, costly, and has poor universality, making it difficult to extend to other scenarios. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a method, device, medium and system for automatic detection of connector foreign objects based on image recognition, eliminating the dependence on a large amount of connector training data, reducing the impact of external interference of the connector and camera shake on detection, realizing reliable assembly of connectors in electronic products, and ultimately achieving stable and effective signal transmission.

[0004] The object of the present invention is achieved through the following solutions: An automatic detection method for connector foreign matter based on image recognition, comprising: Before detection, the connector edge is segmented at the pixel level, and the foreign body judgment process is completed through local image registration and similarity comparison learning.

[0005] Furthermore, the connector edge is segmented at the pixel level before detection, and the foreign body identification process is completed through local image registration and similarity comparison learning, which specifically includes the following sub-steps: Step (1) calibrate the connector outline in the standard connector to obtain the connector template image and fine mask, where the mask outline is represented by the coordinates S_mask of a set of edge contour points; Step (2) determines the connector model and short side orientation, reads the orientation of the standard connector S and the connector to be tested P, and calculates the coordinates of the key position pins based on the orientation information, and uses multiple pairs of coordinates to solve the transformation from S to P, i.e., the affine matrix M: S→P; Step (3): Apply the transformation matrix M to all points S_mask of the mask contour to obtain the target contour P_mask, i.e., the connector contour to be inspected. Draw the target contour and fill it to obtain the mask of the area to be inspected. Use the mask of the area to be inspected to delineate the image to be inspected. Step (4): Use image processing algorithms to synthesize image pairs with differences, pre-train the neural network, and implement the segmentation task of the difference area; Step (5): splice the template image and the image to be inspected together and input them into the network, and the network outputs an anomaly score map; Step (6): output the detection results and images.

[0006] Furthermore, in step (1), the four key position pin coordinates S1, S2, S3, and S4 of the standard connector and the four key position pin coordinates P1, P2, P3, and P4 of the connector to be inspected are obtained from left to right, and these four pairs of coordinates are used to solve the transformation from S to P, that is, the affine matrix M: S→P.

[0007] Furthermore, in step (5), a sub-step is further included: scaling the anomaly score map to 0 to 255 and then visualizing the output.

[0008] Furthermore, the visualization method includes a heat map form.

[0009] A connector foreign body automatic detection device based on image recognition comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method described in any one of the above items is executed.

[0010] A computer-readable storage medium stores a computer program, wherein the computer program executes any of the above methods when loaded by a processor.

[0011] An electronic system includes the above-mentioned automatic connector foreign body detection device based on image recognition.

[0012] The beneficial effects of the present invention include: The present invention performs foreign body detection inside the connector, realizes the reliable assembly of the connector in the electronic product, and ultimately realizes the stable and effective transmission of the signal. Specifically, the present invention provides an automatic detection method for connector foreign bodies based on image recognition, which mainly completes the foreign body judgment process through local alignment and similarity comparison learning of the image. Compared with the current foreign body detection method, this method will effectively reduce the missed detection rate of foreign bodies and reduce false detections caused by interference outside the area to be inspected, and can effectively overcome the shortcomings of traditional connector abnormality detection, such as long cycle, high cost, poor universality, and difficulty in extension to other scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 Schematic diagram of calculating the affine transformation matrix from key pin coordinates; Figure 3 Schematic diagram of pixel-level segmentation of the connector image to be inspected using the transformed mask; Figure 4 Schematic diagram of connector surface foreign body detection through synthetic data pre-training; Figure 5 This is a visualization diagram of the detection results of the present invention on some real data. DETAILED DESCRIPTION

[0015] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0016] The specific implementation process of the present invention is as follows: This invention specifically provides a method for detecting foreign objects in connectors during the assembly of electronic products, specifically automating this detection through image processing and comparison. This method is primarily used for connector assembly inspection in electronic products with high signal transmission requirements. The invention aims to reduce the large amount of data required for initial connector training and mitigate the impact of external connector interference and camera shake on detection. By performing pixel-level segmentation on connector edges prior to inspection, high-precision anomaly detection is achieved. Furthermore, a model is constructed that can rapidly adapt to a wide range of connector models using only a single template image.

[0017] More specifically, in a preferred embodiment, Figure 1 As shown, the following steps are included: (1) Determine the connector model and short side orientation, read the orientation of the standard connector and the connector to be tested, and calculate the coordinates of the key position pins based on the orientation information. From left to right, four pin coordinates can be obtained, S1, S2, S3, S4 (standard connector), P1, P2, P3, P4 (connector to be tested), and use the four pairs of coordinates to solve the transformation from S to P, that is, the affine matrix M: S→P.

[0018] (2) The connector outline is calibrated in the standard connector to obtain the connector template image and fine mask. The mask outline can be represented by the coordinates S_mask of a set of edge contour points.

[0019] (3) Apply the transformation matrix M to all points S_mask of the mask contour to obtain the target contour P_mask. Draw the target contour and fill it to obtain the contour of the area to be inspected. Use the contour of the area to be inspected to delineate the image to be inspected.

[0020] (4) Loading the trained neural network for detecting foreign objects.

[0021] (5) The template image and the image to be inspected are overlapped and input into the network. The network outputs an anomaly score map, which is scaled to 0 to 255 and then visualized in the form of a heat map.

[0022] (6) Output detection results and images.

[0023] Figure 2 Schematic diagram of calculating the affine transformation matrix from key pin coordinates; Figure 3 Schematic diagram of pixel-level segmentation of the connector image to be inspected using the transformed mask; Figure 4 Schematic diagram of connector surface foreign body detection through synthetic data pre-training; Figure 5 This is a visualization diagram of the detection results of the present invention on some real data.

[0024] It should be noted that within the scope of protection defined in the claims of the present invention, the following embodiments can be combined and / or expanded or replaced in any logical way from the above specific implementation methods, such as disclosed technical principles, disclosed technical features or implicitly disclosed technical features.

[0025] Example 1 An automatic detection method for connector foreign matter based on image recognition, comprising: Before detection, the connector edge is segmented at the pixel level, and the foreign body judgment process is completed through local image registration and similarity comparison learning.

[0026] Example 2 Based on Example 1, the connector edge is segmented at the pixel level before detection, and the foreign body identification process is completed through local image registration and similarity comparison learning, which specifically includes the following sub-steps: Step (1) calibrate the connector outline in the standard connector to obtain the connector template image and fine mask, where the mask outline is represented by the coordinates S_mask of a set of edge contour points; Step (2) determines the connector model and short side orientation, reads the orientation of the standard connector S and the connector to be tested P, and calculates the coordinates of the key position pins based on the orientation information, and uses multiple pairs of coordinates to solve the transformation from S to P, i.e., the affine matrix M: S→P; Step (3): Apply the transformation matrix M to all points S_mask of the mask contour to obtain the target contour P_mask, i.e., the connector contour to be inspected. Draw the target contour and fill it to obtain the mask of the area to be inspected. Use the mask of the area to be inspected to delineate the image to be inspected. Step (4): Use image processing algorithms to synthesize image pairs with differences, pre-train the neural network, and implement the segmentation task of the difference area; Step (5): splice the template image and the image to be inspected together and input them into the network, and the network outputs an anomaly score map; Step (6): output the detection results and images.

[0027] Example 3 On the basis of Example 2, in step (1), the four pin coordinates S1, S2, S3, and S4 of the standard connector and the four pin coordinates P1, P2, P3, and P4 of the connector to be inspected are obtained from left to right, and these four pairs of coordinates are used to solve the transformation from S to P, that is, the affine matrix M: S→P.

[0028] Example 4 Based on Example 2, in step (5), a sub-step is further included: scaling the anomaly score map to 0 to 255 and then visualizing the output.

[0029] Example 5 Based on Example 4, the visualization method includes a heat map form.

[0030] Example 6 A connector foreign body automatic detection device based on image recognition includes a processor and a memory, wherein the memory stores a computer program. When the computer program is loaded by the processor, the method described in any one of Examples 1 to 5 is executed.

[0031] Example 7 A computer-readable storage medium stores a computer program, wherein the computer program, when loaded by a processor, executes the method described in any one of Examples 1 to 5.

[0032] Example 8 An electronic system includes the automatic connector foreign body detection device based on image recognition described in Example 6.

[0033] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not limit the units themselves.

[0034] According to one aspect of an embodiment of the present invention, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0035] As another aspect, embodiments of the present invention further provide a computer-readable medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not incorporated into the electronic device. The computer-readable medium carries one or more programs, and when executed by the electronic device, the electronic device implements the methods described in the above embodiments.

Claims

1. A method for automatically detecting foreign matter in connectors based on image recognition, characterized in that: include: Before detection, the connector edge is segmented at the pixel level, and the foreign body judgment process is completed through local image registration and similarity comparison learning.

2. The method for automatically detecting foreign matter in connectors based on image recognition according to claim 1, characterized in that: Before detection, the connector edge is segmented at the pixel level, and the foreign body identification process is completed through local image registration and similarity comparison learning, which specifically includes the following sub-steps: Step (1) calibrate the connector outline in the standard connector to obtain the connector template image and fine mask, where the mask outline is represented by the coordinates S_mask of a set of edge contour points; Step (2) determines the connector model and short side orientation, reads the orientation of the standard connector S and the connector to be tested P, and calculates the coordinates of the key position pins based on the orientation information, and uses multiple pairs of coordinates to solve the transformation from S to P, i.e., the affine matrix M: S→P; Step (3): Apply the transformation matrix M to all points S_mask of the mask contour to obtain the target contour P_mask, i.e., the connector contour to be inspected. Draw the target contour and fill it to obtain the mask of the area to be inspected. Use the mask of the area to be inspected to delineate the image to be inspected. Step (4): Use image processing algorithms to synthesize image pairs with differences, pre-train the neural network, and implement the segmentation task of the difference area; Step (5): splice the template image and the image to be inspected together and input them into the network, and the network outputs an anomaly score map; Step (6): output the detection results and images.

3. The method for automatically detecting foreign matter in connectors based on image recognition according to claim 2, characterized in that: In step (1), the four key position pin coordinates S1, S2, S3, and S4 of the standard connector and the four key position pin coordinates P1, P2, P3, and P4 of the connector to be inspected are obtained from left to right. These four pairs of coordinates are used to solve the transformation from S to P, namely the affine matrix M: S→P.

4. The method for automatically detecting foreign matter in connectors based on image recognition according to claim 2, characterized in that: In step (5), the sub-steps of scaling the anomaly score map to 0 to 255 and then visualizing the output are also included.

5. The method for automatically detecting foreign matter in connectors based on image recognition according to claim 4, characterized in that: Visualization methods include heat maps.

6. An automatic detection device for connector foreign matter based on image recognition, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to any one of claims 1 to 5 is executed.

7. A computer-readable storage medium, characterized in that A computer program is stored in a readable storage medium, and when the computer program is loaded by a processor, the method according to any one of claims 1 to 5 is executed.

8. An electronic system, characterized in that: Including the automatic detection device for connector foreign matter based on image recognition as described in claim 6.

Images