Machine Vision-Based Method and System for Inspecting Electrical Connectors

Abstract

This application relates to the field of electrical connector technology, and in particular to a machine vision-based method and system for electrical connector inspection. The method includes establishing triplet template data and a template structure topology diagram of the electrical connector, and obtaining a binary mask for the electrical connector; generating a topology diagram structure matching value based on the binary mask and the template structure topology diagram; generating a foreground image of the electrical connector and generating a material property matching degree; calculating the matching degree of the topology diagram structure matching value and the material property matching degree, and obtaining a change image of the electrical connector; and generating an electrical connector inspection result based on the change image of the electrical connector. This application, by combining the topology diagram structure matching value and the material property matching degree to calculate the comprehensive matching degree of the electrical connector, can integrate the dual constraints of structural topology and material properties to form a comprehensive assessment of the electrical connector's model attribution and quality level, thereby improving the accuracy of electrical connector inspection.

Description

Technical Field

[0001] This application relates to the field of electrical connector technology, and in particular to a machine vision-based method and system for inspecting electrical connectors. Background Technology

[0002] Electrical connectors are key basic components used to achieve separable connections in electrical circuits. In the electronics manufacturing industry, as a key component for realizing circuit connections, the quality of electrical connectors directly affects the reliability and performance of equipment. Therefore, high-precision testing of electrical connectors is of utmost importance.

[0003] In existing technologies, traditional electrical connector inspection methods mainly rely on manual visual inspection or machine vision inspection based on single geometric features. Manual inspection suffers from drawbacks such as low efficiency, strong subjectivity, and susceptibility to fatigue. In contrast, traditional machine vision inspection methods, such as the invention patent published on July 1, 2018, with announcement number CN108269255A, disclose a machine vision-based electrical connector inspection device and method. The device includes an inspection platform with several inspection stations for placing electrical connectors; a three-axis moving mechanism located on the outer periphery of the inspection platform; and a binocular camera rotatably mounted on the three-axis moving mechanism. At the active end of the mechanism, a binocular camera is used to image the electrical connector placed at the inspection station. The output of the binocular camera is connected to a signal processing mechanism. The signal processing mechanism identifies the features of the pins and sockets on the electrical connector based on the images captured by the binocular camera, and compares the pin and socket features with the features of a set standard template to obtain the comparison result. The binocular camera consists of a left camera and a right camera. The left and right cameras are of the same model and are placed parallel and aligned. The left and right cameras respectively image the pins on the same electrical connector to determine whether the pins are protruding or retracted. Although the above-mentioned patented technical solution can improve inspection efficiency and achieve partial automation, it generally lacks the ability to integrate and analyze the multi-dimensional topological relationships, material properties and geometric features of the electrical connector structure, making it difficult to accurately evaluate the comprehensive performance of the electrical connector in complex industrial scenarios.

[0004] In addition, most existing testing methods only target single geometric dimensions or appearance defects of electrical connectors, lacking joint analysis of spatial relationships between structural units and material properties such as material type and surface reflectivity. This makes it impossible to effectively identify hidden defects caused by structural topological anomalies, and it is difficult to meet the testing requirements of electrical connectors in high-precision and high-reliability scenarios. Summary of the Invention

[0005] Therefore, it is necessary to address the aforementioned technical problems by providing a machine vision-based electrical connector inspection method and system that can calculate the comprehensive matching degree of electrical connectors by combining topological graph structural matching values ​​and material property matching degrees. This system can integrate the dual constraints of structural topology and material properties to form a comprehensive assessment of the electrical connector model attribution and quality level, significantly improving the accuracy, comprehensiveness, and engineering practicality of electrical connector inspection.

[0006] The technical solution of this invention is as follows:

[0007] A machine vision-based method for detecting electrical connectors, the method comprising:

[0008] Establish the triplet template data and template structure topology diagram of the electrical connector, and obtain the binary mask of the electrical connector;

[0009] Generate topology map structure matching values ​​based on the binary mask and template structure topology map of the electrical connector;

[0010] An electrical connector foreground image is generated based on the binary mask and the electrical connector image. Material matching processing is then performed on the electrical connector foreground image to generate a material property matching degree.

[0011] The matching degree of the topology map structure matching value and material property matching degree is calculated, and the electrical connector change image is obtained. The electrical connector detection result is generated based on the electrical connector change image.

[0012] Specifically, this involves establishing the triplet template data and template structure topology of the electrical connector, and obtaining the binary mask of the electrical connector, including:

[0013] After disassembling the electrical connector, establish the ternary template data and template structure topology diagram of the electrical connector. The ternary template data includes: structural data, geometric partition data and material property data.

[0014] An image of the electrical connector is acquired, and edge detection is performed on the electrical connector image to obtain a binary mask of the electrical connector.

[0015] Specifically, after disassembling the electrical connector, a ternary template data and a template structure topology diagram of the electrical connector are established. The ternary template data includes: structural data, geometric partition data, and material property data; including:

[0016] The physical structure of the electrical connector is disassembled to identify its structural units;

[0017] The structural units of the electrical connector are constructed using templates according to the pre-defined ternary structure;

[0018] Each structural unit is assigned an independent structural identifier, and triplet template data and template structural topology diagram are constructed.

[0019] Specifically, it includes:

[0020] Generate an electrical connector topology map based on the aforementioned electrical connector binarization mask;

[0021] The electrical connector topology graph and the template structure topology graph are matched using a subgraph isomorphism algorithm to generate a topology graph structure matching value.

[0022] Specifically, an electrical connector foreground image is generated based on the binary mask and the electrical connector image. Material matching processing is then performed on the electrical connector foreground image to generate a material property matching degree, including:

[0023] The binary mask of the electrical connector and the image of the electrical connector are superimposed, and the foreground region is selected to generate a foreground image of the electrical connector.

[0024] Optical feature analysis is performed on the foreground image of the electrical connector to obtain the material characteristics of the foreground image;

[0025] Calculate the Euclidean distance between the material features and material property data of the foreground image to generate a material property matching degree.

[0026] Specifically, the matching degree of the topology map structure matching value and material property matching degree is calculated, and an electrical connector change image is obtained. Based on the electrical connector change image, an electrical connector detection result is generated, including:

[0027] The matching degree of the topology structure matching value and material property matching degree is calculated to generate the comprehensive matching degree of the electrical connector;

[0028] Based on the overall matching degree of the electrical connector, the triplet template data corresponding to the electrical connector image is obtained;

[0029] Based on the structural data in the triplet template data, a non-rigid transformation is performed on the foreground image of the electrical connector to obtain a transformed image of the electrical connector.

[0030] Based on the image of the electrical connector change, perform image morphology analysis and generate electrical connector detection results.

[0031] Specifically, based on the image of the electrical connector change, morphological analysis is performed, and electrical connector detection results are generated, including:

[0032] Based on the electrical connector test results, image morphology analysis is performed, and local morphology factors are generated;

[0033] Local difference values ​​are generated based on the local morphological factors;

[0034] The electrical connector test results are generated based on the local difference values.

[0035] Specifically, a machine vision-based electrical connector inspection system is also provided, the system comprising:

[0036] The binarization mask acquisition module is used to establish the triplet template data and template structure topology diagram of the electrical connector, and to acquire the binarization mask of the electrical connector.

[0037] The structure matching generation module is used to generate topology map structure matching values ​​based on the binary mask and template structure topology map of the electrical connector;

[0038] The attribute matching generation module is used to generate a foreground image of the electrical connector based on the binary mask of the electrical connector and the image of the electrical connector, and to perform material matching processing on the foreground image of the electrical connector to generate a material attribute matching degree.

[0039] The detection result generation module is used to calculate the matching degree of the topology map structure matching value and material property matching degree, obtain the electrical connector change image, and generate the electrical connector detection result based on the electrical connector change image.

[0040] Specifically, the binarization mask acquisition module is also used for: after disassembling the electrical connector, establishing triplet template data and template structure topology diagram of the electrical connector, wherein the triplet template data includes: structural data, geometric partition data and material property data; acquiring an image of the electrical connector, performing edge detection on the image of the electrical connector, and obtaining a binarization mask of the electrical connector.

[0041] Specifically, the binarization mask acquisition module is also used to: disassemble the physical structure of the electrical connector and identify the structural units of the electrical connector; construct templates for the structural units of the electrical connector according to a preset ternary structure; assign an independent structural identifier to each structural unit and construct ternary template data and template structure topology diagram.

[0042] Specifically, the structure matching generation module is further configured to: generate an electrical connector topology map based on the electrical connector binarization mask; and match the electrical connector topology map and the template structure topology map using a subgraph isomorphism algorithm to generate a topology map structure matching value.

[0043] Specifically, the attribute matching generation module is further configured to: superimpose the binary mask of the electrical connector and the image of the electrical connector and filter out the foreground region to generate a foreground image of the electrical connector; perform optical feature analysis on the foreground image of the electrical connector to obtain the material features of the foreground image; calculate the Euclidean distance between the material features of the foreground image and the material property data to generate a material property matching degree.

[0044] Specifically, the detection result generation module is further used to: calculate the matching degree of the topology map structure matching value and material property matching degree to generate the comprehensive matching degree of the electrical connector; obtain the triplet template data corresponding to the electrical connector image based on the comprehensive matching degree of the electrical connector; perform non-rigid transformation on the foreground image of the electrical connector based on the structural data in the triplet template data to obtain the transformed image of the electrical connector; perform image morphology analysis based on the transformed image of the electrical connector, and generate the detection result of the electrical connector.

[0045] Specifically, the detection result generation module is further configured to: perform image morphology analysis based on the electrical connector detection results and generate local morphology factors; generate local difference values ​​based on the local morphology factors; and generate electrical connector detection results based on the local difference values.

[0046] Optionally, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps described in the machine vision-based electrical connector detection method.

[0047] Optionally, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps described in the machine vision-based electrical connector detection method.

[0048] This invention relates to machine learning and image recognition technologies, and its technical effects are as follows:

[0049] (1) By constructing a standard triplet template containing structural data, geometric partition data, and material property data, and a template structure topology diagram, combined with key technologies such as real-time image acquisition and edge detection, connected component analysis, subgraph isomorphic matching, optical feature analysis, and non-rigid transformation, multi-dimensional detection of the structural topology, material properties, and local morphology of electrical connectors is realized. By calculating the topology diagram structure matching value, material property matching degree, and comprehensive matching degree, and based on the difference analysis between the local morphology factor and the preset threshold, the structural anomalies, material deviations, or morphological defects of electrical connectors can be accurately located, forming a complete detection process from feature modeling to defect warning;

[0050] (2) By matching the topology graph of the electrical connector with the topology graph of the template structure through the subgraph isomorphism algorithm, the spatial relationships (inclusion, adjacency, order, etc.) between various structural units of the electrical connector (such as terminal pins, shell skeleton, positioning buckles, etc.) can be effectively analyzed. The semantic-level verification of the structural topology is achieved through the quantitative matching of nodes and edges (number of successfully matched nodes, percentage of consistent edges). This method breaks through the limitation of traditional detection that only focuses on a single geometric dimension. It can quickly identify hidden defects caused by abnormal layout of structural units or incorrect connection relationships, ensuring the overall structural integrity and functional reliability of the electrical connector.

[0051] (3) By calculating the Euclidean distance between the material features of the foreground image and the material property data, the matching degree of material properties can be obtained, which enables quantitative analysis of the material type (such as phosphor bronze, nylon) and physical properties (surface roughness, reflectivity) of each structural unit of the electrical connector. This process transforms optical features (reflectivity, contrast, reflectance, etc.) into comparable numerical indicators, realizing the leap from qualitative description to quantitative detection of material properties, and can accurately identify performance risks caused by material deviations or surface treatment defects. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a machine vision-based electrical connector inspection method in one embodiment.

[0053] Figure 2 This is a structural block diagram of a machine vision-based electrical connector inspection system in one embodiment;

[0054] Figure 3 This is a structural block diagram of a computer device in one embodiment. Detailed Implementation

[0055] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0056] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0057] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0059] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0061] In one embodiment, a terminal is provided, the terminal being configured to: establish triplet template data and template structure topology map of an electrical connector, and obtain a binary mask of the electrical connector; generate a topology map structure matching value based on the binary mask of the electrical connector and the template structure topology map; generate an electrical connector foreground image based on the binary mask of the electrical connector and the electrical connector image, perform material matching processing on the electrical connector foreground image, and generate a material property matching degree; calculate the matching degree of the topology map structure matching value and the material property matching degree, and obtain a change image of the electrical connector, and generate an electrical connector detection result based on the change image of the electrical connector.

[0062] The terminal may be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices.

[0063] In one embodiment, such as Figure 1 As shown, a machine vision-based method for detecting electrical connectors is provided, the method comprising:

[0064] Step S100: Establish the triplet template data and template structure topology diagram of the electrical connector, and obtain the binary mask of the electrical connector;

[0065] Step S200: Generate topology map structure matching values ​​based on the binary mask and template structure topology map of the electrical connector;

[0066] Step S300: Generate a foreground image of the electrical connector based on the binary mask of the electrical connector and the image of the electrical connector; perform material matching processing on the foreground image of the electrical connector to generate a material property matching degree.

[0067] Step S400: Calculate the matching degree of the topology map structure matching value and material property matching degree, and obtain the electrical connector change image. Generate the electrical connector detection result based on the electrical connector change image.

[0068] In this application, a triplet template data and template structure topology diagram of the electrical connector are established, and a binary mask of the electrical connector is obtained. A topology diagram structure matching value is generated based on the binary mask and the template structure topology diagram. A foreground image of the electrical connector is generated, and a material property matching degree is generated. The matching degree of the topology diagram structure matching value and the material property matching degree is calculated, and a change image of the electrical connector is obtained. An electrical connector detection result is generated based on the change image. This application calculates the comprehensive matching degree of the electrical connector by combining the topology diagram structure matching value and the material property matching degree. This integrates the dual constraints of structural topology and material properties, forming a comprehensive assessment of the electrical connector model attribution and quality level, thus improving the accuracy of electrical connector detection.

[0069] In one embodiment, step S100: establishing the triplet template data and template structure topology diagram of the electrical connector, and obtaining the binary mask of the electrical connector, includes:

[0070] Step S110: After disassembling the electrical connector, establish the triplet template data and template structure topology diagram of the electrical connector. The triplet template data includes: structural data, geometric partition data and material property data.

[0071] Step S120: Obtain an image of the electrical connector, perform edge detection on the image of the electrical connector, and obtain a binary mask of the electrical connector.

[0072] In this embodiment, in order to establish the triplet template data and template structure topology map of the electrical connector, the triplet template data and template structure topology map of the electrical connector are established after disassembling the electrical connector. The triplet template data includes: structural data, geometric partition data and material property data. Then, in order to perform topology comparison later, it is necessary to generate a binary mask of the electrical connector. Therefore, by acquiring the image of the electrical connector, edge detection is performed on the image of the electrical connector to obtain the binary mask of the electrical connector.

[0073] In one embodiment, step S110: After disassembling the electrical connector, establish the ternary template data and template structure topology diagram of the electrical connector. The ternary template data includes: structural data, geometric partition data, and material property data; including:

[0074] Step S110: Disassemble the physical structure of the electrical connector and identify its structural units;

[0075] Step S120: Construct the structural units of the electrical connector using a template according to the preset ternary structure;

[0076] Step S130: Assign an independent structural identifier to each structural unit, and construct triplet template data and template structure topology diagram.

[0077] In this embodiment, the physical structure of the electrical connector is first disassembled to identify and define key structural units such as terminal pins, housing skeleton, and positioning clips, which serve as the basis for template modeling. A typical electrical connector sample, such as the JSTPH series 2.0mm pitch connector, is manually disassembled to separate components such as terminal pins, nylon housing, and metal clips. Basic dimensions such as pin diameter and housing wall thickness are measured using precision calipers. Typical values ​​for pin diameter are 0.3-0.6mm, and typical values ​​for housing wall thickness are 0.8-1.2mm.

[0078] Next, the structural unit of each electrical connector is templated according to the preset ternary structure of "structural region + geometric partition data + material property data".

[0079] The structural area clearly defines the connections between structural units; for example, the distance between terminal pins and positioning clips should be less than 2mm. The geometric area extracts parameters such as shape, size, and spatial position of each unit and sets tolerance ranges, such as pin diameter, housing aspect ratio, and pin spacing. Material properties indicate material type and physical characteristics, such as phosphor bronze for metal elastic components, nylon for insulating injection molded bodies, and their surface roughness. Each structural unit is then assigned an independent structural identifier, binding function, geometry, and material properties with the identifier to form structured data. It should be noted that the "structural area + geometric area + material properties" ternary structure is used to construct each connector model. For example, for the terminal pins of an electrical connector; the structural area specifies that the distance between terminal pins and positioning clips should be less than 2mm. Geometric areas include rectangles / circles with diameters of 0.3-1.0mm, lengths of 3-10mm, and array spacing accuracy of ±0.05mm. Material properties include reflectivity, contrast, and reflectance.

[0080] Then, a template structure topology graph is constructed to represent the spatial relationships between various structural features of the electrical connector. Nodes represent structural units of the electrical connector, and edges describe relationships such as inclusion, adjacency, and order, along with parameter constraints. Finally, the triplet template data and the template structure topology graph form a template database. Typical models are matched and verified, and parameters are adjusted to ensure the robustness of the templates, providing semantic-level structural constraints and prior knowledge for subsequent image reconstruction and detection.

[0081] In one embodiment, step S120: acquiring an image of the electrical connector, performing edge detection on the electrical connector image, and obtaining a binary mask for the electrical connector, as follows:

[0082] First, edge detection is performed on the electrical connector image using the Sobel operator to find closed regions, such as rectangular housings, circular pins, and trapezoidal buckles. Each region is regarded as a "node", and each node is labeled with a preliminary semantic label, such as "suspected housing" or "suspected pin".

[0083] Further, the steps for edge detection of the electrical connector image using the Sobel operator are as follows:

[0084]

[0085] in, This represents the gradient value of the electrical connector image in the horizontal direction at coordinates (x, y). This represents the gradient value of the electrical connector image in the vertical direction at coordinates (x, y). The pixel value of the electrical connector image at coordinates (x, y); The horizontal gradient operator of the Sobel operator; This is the vertical gradient operator for the Sobel operator.

[0086] It should be noted that the Sobel operator is an image processing operator used for edge detection. It is computationally efficient, practical in real time, and can improve computational efficiency when processing large-scale images.

[0087] Specifically, the Sobel operator sets the weights of the left (-1) and right (+1) pixels of the convolution kernel to opposite values. If the left pixel has a larger value (i.e., a dark area) and the right pixel has a smaller value (i.e., a bright area), it results in a larger positive value, and vice versa. The weight of the middle row is 0, meaning the current pixel value does not directly affect the result. The purpose of this design is to focus only on the difference between the left and right pixels, without considering the current pixel. The weights of the upper and lower rows (-1 and 1) are equal, but the weight of the middle row is twice that of the upper and lower rows (-2 and 2). This means that the upper and lower rows have a greater impact on the pixels.

[0088] Based on the horizontal and vertical gradient values, calculate the gradient magnitude and corresponding gradient direction of the electrical connector image:

[0089] ;

[0090] ;

[0091] in, This represents the gradient magnitude of the electrical connector image at coordinates (x, y). The gradient direction of the electrical connector image at coordinates (x, y); This represents the gradient value of the electrical connector image in the horizontal direction at coordinates (x, y). This represents the gradient value of the electrical connector image in the vertical direction at coordinates (x, y).

[0092] For the gradient magnitude and gradient direction at coordinates (x, y) in the electrical connector image, the edge intensity image NMS(x, y) is obtained using non-maximum suppression. Specifically, the gradient direction of the electrical connector image at coordinates (x, y) is... Discretized into four main directions :

[0093]

[0094] in, These are the four main directions after discretization. Let (x, y) be the gradient direction of the electrical connector image at coordinates (x, y).

[0095] The neighboring pixels to be compared are determined by the four main discretization directions:

[0096]

[0097] in, Let x and y be the coordinates of two adjacent pixels in the electrical connector image at coordinates (x, y), where x and y are the horizontal and vertical coordinates, respectively. These are the four main directions after discretization.

[0098] Perform nonmaximum suppression calculation:

[0099]

[0100] in, For edge intensity images, Let be the gradient magnitude of the electrical connector image at coordinates (x, y). and Let be the gradient magnitude of two adjacent pixels in the electrical connector image at coordinates (x, y).

[0101] Let (x, y) be the coordinates of two adjacent pixels in the electrical connector image.

[0102] Morphological operations are used to process edge intensity images to remove noise, fill small holes, connect broken edges, and obtain the connected components of the edge intensity image. Morphological operations include, but are not limited to, traditional methods such as dilation, erosion, and opening operations.

[0103] Binarizing the edge intensity image yields a binary mask for the electrical connector:

[0104]

[0105] in, For binary masking, This is an edge intensity image.

[0106] It should be noted that the connected component of the edge intensity image is also the connected component of the binarization mask. Therefore, this application performs binarization processing based on the result obtained after non-maximum suppression, dividing the pixel values ​​of the image into 0 and 1, changing all pixel values ​​contained in the connected component of the edge intensity to 1, and changing the remaining non-connected pixels to 0.

[0107] In one embodiment, step S200: generating a topology map structure matching value based on the binary mask and template structure topology map of the electrical connector; including:

[0108] Step S210: Generate an electrical connector topology map based on the binary mask of the electrical connector;

[0109] Step S220: Match the electrical connector topology diagram and the template structure topology diagram using a subgraph isomorphism algorithm to generate a topology diagram structure matching value.

[0110] In this embodiment, for each connected region of the binarized mask, each region of the electrical connector is identified by the contour features of the connected region, namely its aspect ratio and roundness. First, the aspect ratio of the connected region is calculated:

[0111]

[0112] Where AR is the aspect ratio of the connected component, W is the width of the connected component, and H is the height of the connected component. It should be noted that for each connected component (the region composed of adjacent target pixels) in the binarized mask, when calculating the aspect ratio of the connected component, the smallest rectangle that exactly contains all pixels within that connected component is obtained. The width of this rectangle is the horizontal distance between its left and right boundaries, and the height is the vertical distance between its top and bottom boundaries. The width and height of this rectangle are used as the width and height of the connected component to calculate its aspect ratio.

[0113] Next, calculate the roundness of the connected components:

[0114]

[0115] Where C is the roundness of the connected region, P is the perimeter of the connected region, and A is the area of ​​the connected region.

[0116] It should be noted that the perimeter of a connected region is the total Euclidean length of the connected region outline chain code, and the area of ​​a connected region is the total number of pixels contained in the connected region.

[0117] Furthermore, the region of the electrical connector corresponding to each connected component is determined by the morphological characteristics of that component. The aspect ratio, roundness, and area determine the type of the connected component. The connected components are first roughly divided to reduce the influence of noise; for example, the area A is made small (e.g., <1000 pixels), and the aspect ratio is... Large connected regions (e.g., 3-10 pixels) and low roundness (e.g., 0.2-0.4 pixels) are defined as the terminal pin area of ​​the electrical connector; connected regions with moderate area A (e.g., 1000-5000 pixels), moderate aspect ratio AR (e.g., 1.5-3), and moderate roundness (e.g., 0.4-0.7 pixels) are defined as the snap-fit ​​area; connected regions with high area A (e.g., >5000 pixels), moderate aspect ratio AR (1-3), and high roundness C (e.g., 0.7-0.9 pixels) are defined as the housing area. Further subdivisions are then made; for example, for the housing area, regions with roundness >0.8 and aspect ratio... The shell region of 1.2 is divided into cylindrical shells, and the shell region with roundness <0.8 and aspect ratio >1.2 is divided into rectangular shells.

[0118] By performing connected component analysis on the binary mask of the electrical connector, a topology graph of the electrical connector is constructed. For example, a large rectangular outline (node ​​A, labeled "shell"), eight small circular outlines (nodes B1-B8, labeled "pin"), and a trapezoidal outline (node ​​C, labeled "clasp") are detected in the binary mask.

[0119] Then, the spatial relationship parameters between nodes are calculated as attributes of "edges". For example, for containment relationships, if all circular nodes are located inside rectangular nodes, edge A-Bi (i=1-8) is marked "Containment". For adjacency relationships, if the distance between the right edge of a trapezoidal node and a rectangular node is <0.5mm, edge AC is marked "Right Adjacent". Also, for arrangement relationships, specifically, if the 8 circular nodes have equal lateral spacing, edge Bi-Bi+1 is marked "Equally Distant Linear Arrangement". Nodes in the electrical connector topology diagram and template structure topology diagram are grouped according to node type.

[0120] Next, candidate matching pairs are constructed, specifically by generating a list of nodes in the electrical connector topology graph that match the type for each node in the template structure topology graph.

[0121] Then, a backtracking search for matching paths is performed. Specifically, starting from the root node of the template structure topology graph (e.g., starting from the shell), the nodes of the template structure topology graph are recursively mapped to the nodes of the electrical connector topology graph, and the connection relationships of the edges are verified to be consistent. For example, if the spacing error between two pins in the electrical connector topology graph and the template structure topology graph is less than a threshold, the recursive mapping is performed starting from the root node (e.g., the shell node) and a depth-first traversal is used: first, electrical connector nodes of the type corresponding to the template root node (e.g., all connected components marked "shell") are selected as initial candidates; for each candidate node, its child nodes (e.g., the shell package) are recursively verified. The system checks whether the terminal pins (including the connector pins) have corresponding nodes in the electrical connector topology diagram that match the type and have consistent spatial relationships. For example, it checks whether the pin node is located inside the candidate housing (inclusion relationship), whether the distance between adjacent clips is <0.5mm (adjacency relationship), and whether the pin spacing is equal (sequential relationship). If a branch fails to verify, it backtracks to the upper-level node to replace the candidate. If all child nodes are successfully mapped and the edge relationship error is within the threshold (such as pin spacing tolerance ±0.05mm), a complete matching path is formed. Finally, the system calculates the proportion of successfully mapped nodes and the proportion of edges with consistent relationships to generate the topology diagram structure matching value (PP value).

[0122] Next, the topology graph of the electrical connector and the topology graph of the template structure are matched using a subgraph isomorphism algorithm to obtain the topology graph structure matching value:

[0123]

[0124] Where PP is the topology graph structure matching value. This indicates the number of nodes that were successfully matched. This represents the total number of points in the template structure topology diagram. To ensure that the number of edges is consistent across nodes. The total number of edges in the template structure topology graph. Match weights to nodes. Weights are matched to edge relationships. + =1.

[0125] It should be noted that when structural integrity is more important than spatial relationships, for example, missing pins / housing is more serious than mismatched spacing. 0.7 is acceptable. A value of 0.3 is acceptable; when spatial relation > structural integrity, such as when matching high-precision connectors, 0.3 is acceptable. A value of 0.7 can be used; when structural integrity equals spatial relationship, such as in automotive electrical connectors (where both structure and assembly must be guaranteed), then... and 0.5 is acceptable for both.

[0126] It's worth noting that the subgraph isomorphism algorithm is used to match the electrical connector topology graph with the template structure topology graph and obtain the topology graph structure matching value, enabling semantic-level verification and quantitative evaluation of the spatial relationships of the electrical connector structural units. Specifically, the template structure topology graph constructs a standard structural model of the electrical connector through nodes (such as terminal pins, housing frames, and other structural units) and edges (such as spatial relationships and parameter constraints like inclusion, adjacency, and sequence), while the electrical connector topology graph extracts the structural units and their spatial relationships of the actual detection object based on connected component analysis of real-time images.

[0127] By using the recursive matching and constraint verification of the subgraph isomorphism algorithm (such as node type matching, edge connection relationship and parameter error threshold judgment), it is possible to accurately analyze whether there are problems such as missing structural units, abnormal layout or incorrect connection relationship in actual electrical connectors, such as pins not being included in the housing or buckle adjacent distance exceeding the tolerance.

[0128] The topology graph structure matching value (PP value) quantifies structural consistency by the ratio of the number of successfully matched nodes to the number of consistent edges. It provides a key indicator of the structural topology dimension for subsequent comprehensive matching degree calculation, enabling the inspection process to not only identify single geometric dimension deviations, but also to determine the design compliance of electrical connectors from the overall level of "structural unit-spatial relationship". This effectively makes up for the shortcomings of traditional inspection methods in missing hidden structural defects and lays a structured analysis foundation for multi-dimensional quality assessment.

[0129] In one embodiment, step S310: generating an electrical connector foreground image based on the binary mask and the electrical connector image, performing material matching processing on the electrical connector foreground image to generate a material property matching degree, including:

[0130] Step S310: Overlay the binary mask of the electrical connector and the image of the electrical connector and filter out the foreground region to generate a foreground image of the electrical connector;

[0131] Step S320: Perform optical feature analysis on the foreground image of the electrical connector to obtain the material features of the foreground image;

[0132] Step S330: Calculate the Euclidean distance between the material features and material property data of the foreground image, and generate the material property matching degree.

[0133] In this embodiment, the binary mask of the electrical connector and the image of the electrical connector are superimposed, and the foreground region is selected to obtain the foreground image of the electrical connector:

[0134]

[0135] in, Foreground image of an electrical connector. For binary masking, The pixel of the electrical connector image at coordinates (x, y).

[0136] It should be noted that the foreground region selected by superimposing the binarized mask and the electrical connector image is also a connected component region of the binarized mask. The foreground region of the electrical connector image has color, while the pixel values ​​of the connected component regions of the binarized mask are all 1. Furthermore, it should be noted that the core significance of superimposing the electrical connector binarized mask and the electrical connector image to select the foreground region and obtain the electrical connector foreground image lies in achieving accurate segmentation of the target area of ​​the electrical connector from the background. The binarized mask, through edge detection and morphological operations, has clearly defined the outline of the electrical connector and the connected components of each structural unit (such as the shell, pins, and latches). The areas with a pixel value of 1 correspond to the actual physical structure of the electrical connector. After superimposing it with the original electrical connector image, the foreground region of the electrical connector can be accurately extracted from the original image based on the connected component range of the mask, removing irrelevant background noise. This allows subsequent optical feature analysis of the foreground image (such as material feature extraction of reflectivity, contrast, and reflectance) to focus on the structural units of the electrical connector itself, avoiding feature deviations caused by background interference.

[0137] Meanwhile, by combining the constructed electrical connector topology map, each region in the foreground image can be clearly labeled as a specific structural unit (such as "pin" or "shell"), thus forming a semantic-level correspondence with the material properties in the standard triplet template data (such as phosphor bronze gold plating and the surface reflectivity of nylon materials). This lays the foundation for calculating the Euclidean distance between the material features of the foreground image and the template material property data, and for achieving accurate matching of material properties.

[0138] In summary, this step is a crucial link between image preprocessing and material detection, ensuring the relevance and accuracy of material analysis, thereby improving the reliability of the overall detection process.

[0139] Based on the electrical connector topology diagram, the type of each foreground region in the foreground image of the electrical connector (such as terminal pins, snap-fits, etc.) is determined. By performing optical feature analysis on each foreground region of the electrical connector foreground image, the material characteristics of the foreground image are obtained. , =[ ], i=1,2,3,..., j=1,2,3,..., , For the j-th foreground region of the electrical connector foreground image, the i-th material feature is... This represents the total number of material features in the foreground region. This represents the total number of foreground regions.

[0140] Optical feature analysis refers to the process of quantitatively extracting the surface optical properties of each classified foreground region (such as terminal pins, housing skeleton, positioning buckles, etc.) in the foreground image of the electrical connector. First, under a standard light source environment (usually a 45° ring LED light source), the maximum reflectivity of the metal terminal pin region (the 95th percentile of the region's gray value, reflecting the gloss of the metal coating), the gray standard deviation of the plastic housing region (characterizing the uniformity of the injection molding texture), and the reflectivity gradient of the buckle edge (the rate of gray change along the normal direction, judging the degree of surface oxidation) are calculated. At the same time, the material type is analyzed by combining the saturation histogram of the HSV color space (such as phosphor bronze presenting a brass hue, and nylon showing a low saturation gray tone). Finally, optical parameters such as reflectivity, texture contrast, and reflectivity distribution are integrated into a dimension-normalized feature vector, providing quantifiable physical property indicators for subsequent material property matching.

[0141] It should be noted that optical feature analysis is performed on each foreground region of the foreground image of the electrical connector, and the optical feature analysis includes the reflectivity, contrast and reflectance of each foreground region.

[0142] Next, by calculating the Euclidean distance between the foreground image material features and the material property data, the material property matching degree is obtained as follows:

[0143]

[0144] Wherein, CL represents the material property matching degree. Let be the total number of foreground regions, and j be the index of the j-th foreground region. Let be the total number of material features in the foreground region, and let i be the index of the material features in the foreground region. For the j-th foreground region of the electrical connector foreground image, the i-th material feature is... Let i be the material feature of the j-th foreground region of the material property data.

[0145] It should be noted that the Material Property Matching (CL) achieves a quantitative assessment of material deviation through multi-level mapping: First, the Euclidean distance between the optical feature vector (containing n1-dimensional parameters such as reflectivity, contrast, and reflectance) of each foreground region (e.g., terminal pins) and the template material property data is calculated to obtain the material deviation of a single region; then, the arithmetic mean of the distance values ​​of all foreground regions (n2 in total) is calculated to obtain the overall material deviation; then, the deviation is nonlinearly mapped to the (0, π / 2) interval using the arctangent function to suppress extreme value disturbances; finally, the output is normalized to the (0, 1) range by multiplying by the coefficient 2 / π, and the reciprocal is taken, where CL approaching 1 indicates that the material fully meets the standard (e.g., the reflectivity of phosphor bronze coating meets the standard), and CL approaching 0 indicates serious deviation (e.g., abnormal reflectivity caused by surface oxidation), realizing a scientific conversion from multi-dimensional optical features to a single quality score.

[0146] In one embodiment, step S400: Calculate the matching degree of the topology map structure matching value and material property matching degree, and obtain an electrical connector change image; generate an electrical connector detection result based on the electrical connector change image, including:

[0147] Step S410: Calculate the matching degree of the topology diagram structure matching value and material property matching degree to generate the comprehensive matching degree of the electrical connector;

[0148] Step S420: Based on the overall matching degree of the electrical connector, obtain the triplet template data corresponding to the electrical connector image;

[0149] Step S430: Based on the structural data in the triplet template data, perform non-rigid transformation on the foreground image of the electrical connector to obtain a transformed image of the electrical connector;

[0150] Step S440: Perform image morphology analysis based on the electrical connector change image and generate electrical connector detection results.

[0151] In this embodiment, the overall matching degree of the electrical connector is calculated by combining the topology diagram structure matching value and the material property matching degree:

[0152]

[0153] Where ZP represents the overall compatibility of the electrical connector, PP represents the topology matching value, and CL represents the material property matching degree.

[0154] This embodiment uses the constructed electrical connector comprehensive matching degree ZP to represent the risk quantifier of defect synergy effect.

[0155] Specifically, the numerator (PP*CL) requires both structure and material to meet standards simultaneously to have basic matching value, emphasizing that defects in either dimension will significantly reduce product reliability. The denominator constructs a dual defect penalty mechanism by introducing the term (1-PP)(1-CL). When there is structural mismatch (low PP) and material abnormality (low CL), this term will increase sharply (up to 1), causing the ZP value to plummet. This simulates the multiplied risk of cascading failures caused by combined defects in industrial scenarios. Adding PP*CL to the denominator maintains the formula's stable convergence and prevents numerical explosion. For example, if PP is 0.99 and CL is 0.99, without adding PP*CL to the denominator, then... .

[0156] Therefore, this type of nonlinear design maintains tolerance when a single defect exists, such as ZP being 0.94 when PP is 0.9 and CL is 1, while imposing an exponential penalty on double defects, such as ZP being 0.66 when PP is 0.7 and CL is 0.7, so that the evaluation results directly map the failure probability of the electrical connector under complex operating conditions.

[0157] Next, the triplet template data corresponding to the electrical connector image with the highest overall matching degree is selected, and the foreground image of the electrical connector is subjected to non-rigid transformation based on the length ratio of the geometric data in the triplet template data.

[0158] Specifically, the feature point set P of the foreground image of the electrical connector is defined as follows: , Let N be the o-th feature point in the foreground image of the electrical connector, and N be the total number of feature points. The geometric data in the triplet template data contains the ideal distance set of the feature points. , Let M represent the ideal distance between the k-th pair of feature points, and M be the total number of ideal distances between feature point pairs in the template feature point ideal distance set. The length ratio constraint is calculated as follows: ,in, The length ratio of the kth feature point pair and This is the kth pair of feature points in the template feature point set.

[0159] It should be noted that the feature point set of the foreground image of the electrical connector is a set of key location points extracted from the foreground image of the electrical connector. These feature points correspond to the salient geometric features of each structural unit of the electrical connector (such as terminal pin endpoints, housing corners, positioning latch edges, etc.), and are used to characterize the actual structural form of the electrical connector. The feature point set is obtained by annotating and extracting key locations such as the contours and corners of each structural unit in the foreground image. Each feature point is represented by coordinates (…). The feature point represents its position in the image, and N is the total number of feature points. These feature points not only carry the spatial position information of each structural unit of the electrical connector, but also form a correspondence with the geometric data in the standard triplet template data, serving as the basic input for non-rigid transformation. During the thin plate spline transformation, the feature point set is compared with the ideal distance set of template feature points to calculate the length ratio constraint, driving the deformation function to nonlinearly adjust the foreground image, so that the detected object and the standard template achieve high-precision alignment in structural morphology, thereby eliminating the interference of factors such as image acquisition perspective and deformation on detection accuracy, and providing a unified benchmark for subsequent local morphology analysis.

[0160] Furthermore, the deformation function of the thin-plate spline transformation method:

[0161]

[0162] in, Let u(p) be the deformation function, representing the mapping of point p in the foreground image of the electrical connector to the target position p + u(p), where u(p) is the displacement vector, and u(p) = ( (p), (p)), (p) represents the elastic displacement of point p in the x-axis direction. (p) represents the nonlinear distortion of point p in the y-axis direction. Let U be the coefficient vector of the o-th control point, U() be the radial basis function, N be the total number of feature points, and o be the index of the o-th feature point.

[0163] It should be noted that the deformation function of the thin-plate spline transformation method simulates the non-rigid deformation of the image as the elastic bending behavior of a thin metal plate, constructing the displacement field by minimizing the bending energy (i.e., the integral of the square of the second derivative); the first term on the right side of the equation Representing the original coordinates (deformation datum), the second displacement vector u(p) is formed by the linear superposition of radial basis functions U(), where It is the control point coefficient vector, which is obtained by solving the distance constraints between feature point pairs (such as the length ratio of pin spacing to template geometry data) and the bending energy minimization objective function to ensure that the relative positions of adjacent structural units (such as terminal pin spacing) are consistent with the template after transformation.

[0164] Specifically, the energy objective function is as follows:

[0165]

[0166] in, The objective function is to minimize energy. This is the bending stiffness regularization coefficient, with a default value of 0.1. M is the total number of ideal distances between feature point pairs in the template feature point ideal distance set. This represents the ideal distance between the k-th pair of feature points. and For the k-th pair of feature points in the template feature point set, and For the k-th pair of feature points in the foreground image of the electrical connector, This represents the points in the foreground image of the electrical connector. Mapped to target location +u( ), Let be the deformation function, representing the points in the foreground image of the electrical connector. Mapped to target location +u( ), This represents the second-order partial derivative of the displacement vector in the x-direction. Let be the mixed second-order partial derivative of the displacement vector u with respect to x and y. Let represent the second-order partial derivative of the displacement vector in the y-direction. The nonlinear and affine coefficients of the deformation function are solved using the least squares method while minimizing the energy objective function.

[0167] The final result is: = ,in, Image showing changes in the electrical connector. This represents the mapping change of the foreground image of the electrical connector through the deformation function.

[0168] Specifically, This represents the distance between the k-th pair of feature points in the electrical connector image after thin-plate spline transformation. and Squaring the difference is a nonlinear amplification of the deviation.

[0169] In one embodiment, step S440: performing image morphology analysis based on the electrical connector change image and generating electrical connector detection results, including:

[0170] Step S441: Perform image morphology analysis based on the electrical connector detection results and generate local morphology factors;

[0171] Step S442: Generate local difference values ​​based on the local morphological factors;

[0172] Step S443: Generate electrical connector detection results based on the local difference values.

[0173] In this embodiment, for each foreground region in the image of the electrical connector change, the local morphological factor of the foreground region is calculated. The local morphological factor includes the curvature, symmetry, etc. of the foreground region.

[0174] The local difference value is obtained by calculating the difference between the local morphological factor and the geometric partition data:

[0175]

[0176] in, Let j be the local difference value of the j-th foreground region. Let be the local morphological factor of the j-th foreground region in the image of the electrical connector variation. Let j be the local morphological factor of the geometric partitioned data. It is the inverse of the covariance matrix of the local morphological factors of the electrical connector image and the local morphological factors of the geometric partition data.

[0177] When the local difference value exceeds a preset threshold, the system will trigger a multi-level early warning mechanism: firstly, it will issue a real-time warning through audible and visual signals, and simultaneously mark the abnormal areas in the electrical connector change image in the operation interface in the form of a visual heat map, accurately locating the specific structural unit (such as terminal pins, housing skeleton, or positioning buckles). Combining the topology diagram information, the system can further analyze the defect type. For example, if the local difference value in the terminal pin area exceeds the limit, it can be determined as pin deformation; if the local difference value in the housing area exceeds the standard, it will indicate the presence of injection molding defects or assembly misalignment, and automatically generate a detection report containing the defect location and type, providing accurate basis for subsequent rework, process adjustment, or quality traceability, realizing a closed-loop process from defect detection to location analysis, cause tracing, and decision support, significantly improving the intelligence and precision of electrical connector detection.

[0178] Therefore, this application presents a machine vision-based method for detecting electrical connectors. By constructing a standard triplet template containing structural data, geometric partitioning data, and material property data, along with a template structure topology graph, and combining key technologies such as real-time image acquisition and edge detection, connected component analysis, subgraph isomorphic matching, optical feature analysis, and non-rigid transformation, it achieves multi-dimensional detection of the electrical connector's structural topology, material properties, and local morphology. By calculating the topology graph's structural matching value, material property matching degree, and comprehensive matching degree, and based on the difference analysis between local morphological factors and preset thresholds, it can accurately locate structural anomalies, material deviations, or morphological defects in the electrical connector, forming a complete detection process from feature modeling to defect early warning.

[0179] Then, a subgraph isomorphism algorithm is used to match the electrical connector topology graph with the template structure topology graph. This effectively resolves the spatial relationships (containment, adjacency, order, etc.) between various structural units of the electrical connector (such as terminal pins, housing skeleton, positioning clips, etc.). Semantic-level verification of the structural topology is achieved through quantitative matching of nodes and edges (number of successfully matched nodes, percentage of consistent edges). This method overcomes the limitations of traditional detection methods that only focus on a single geometric dimension, and can quickly identify hidden defects caused by abnormal structural unit layout or incorrect connection relationships, ensuring the overall structural integrity and functional reliability of the electrical connector.

[0180] Next, by calculating the Euclidean distance between the material features of the foreground image and the material property data, the matching degree of material properties is obtained. This enables quantitative analysis of the material type (such as phosphor bronze, nylon) and physical properties (surface roughness, reflectivity) of each structural unit of the electrical connector. This process transforms optical characteristics (reflectivity, contrast, reflectance, etc.) into comparable numerical indicators, realizing a leap from qualitative description to quantitative detection of material properties. It can accurately identify performance risks caused by material deviations or surface treatment defects, providing a scientific basis for the material consistency testing of electrical connectors.

[0181] Finally, by combining the topological matching value and material property matching degree to calculate the comprehensive matching degree of the electrical connector, the dual constraints of structural topology and material properties can be integrated to form a comprehensive assessment of the connector's model classification and quality level. Based on the comprehensive matching degree, the optimal template is selected and non-rigid transformation is performed, eliminating the influence of factors such as image acquisition perspective and deformation on detection accuracy, enabling high-precision alignment between the inspected object and the standard template. Further local morphological analysis, through the calculation of differences in factors such as curvature and symmetry, can achieve precise location and degree quantification of defects, ultimately constructing an intelligent detection closed loop from "multi-dimensional feature matching" to "defect location early warning," significantly improving the accuracy, comprehensiveness, and engineering practicality of electrical connector inspection.

[0182] In one embodiment, such as Figure 2 As shown, a machine vision-based electrical connector inspection system is also provided, the system comprising:

[0183] The binarization mask acquisition module is used to establish the triplet template data and template structure topology diagram of the electrical connector, and to acquire the binarization mask of the electrical connector.

[0184] The structure matching generation module is used to generate topology map structure matching values ​​based on the binary mask and template structure topology map of the electrical connector;

[0185] The attribute matching generation module is used to generate a foreground image of the electrical connector based on the binary mask of the electrical connector and the image of the electrical connector, and to perform material matching processing on the foreground image of the electrical connector to generate a material attribute matching degree.

[0186] The detection result generation module is used to calculate the matching degree of the topology map structure matching value and material property matching degree, obtain the electrical connector change image, and generate the electrical connector detection result based on the electrical connector change image.

[0187] In one embodiment, the binarization mask acquisition module is further configured to: after disassembling the electrical connector, establish triplet template data and template structure topology diagram of the electrical connector, wherein the triplet template data includes: structural data, geometric partition data and material property data; acquire an electrical connector image of the electrical connector, perform edge detection on the electrical connector image, and obtain a binarization mask of the electrical connector.

[0188] In one embodiment, the binarization mask acquisition module is further configured to: disassemble the physical structure of the electrical connector and identify the structural units of the electrical connector; construct templates for the structural units of the electrical connector according to a preset ternary structure; assign an independent structural identifier to each structural unit and construct ternary template data and template structure topology diagram.

[0189] In one embodiment, the structure matching generation module is further configured to: generate an electrical connector topology map based on the electrical connector binarization mask; and match the electrical connector topology map and the template structure topology map using a subgraph isomorphism algorithm to generate a topology map structure matching value.

[0190] In one embodiment, the attribute matching generation module is further configured to: superimpose the binary mask of the electrical connector and the image of the electrical connector and filter out the foreground region to generate a foreground image of the electrical connector; perform optical feature analysis on the foreground image of the electrical connector to obtain the material features of the foreground image; and calculate the Euclidean distance between the material features of the foreground image and the material property data to generate a material property matching degree.

[0191] In one embodiment, the detection result generation module is further configured to: calculate the matching degree of the topology map structure matching value and material property matching degree to generate a comprehensive matching degree of the electrical connector; obtain the triplet template data corresponding to the electrical connector image based on the comprehensive matching degree of the electrical connector; perform non-rigid transformation on the foreground image of the electrical connector based on the structural data in the triplet template data to obtain a transformed image of the electrical connector; perform image morphology analysis based on the transformed image of the electrical connector, and generate an electrical connector detection result.

[0192] In one embodiment, the detection result generation module is further configured to: perform image morphology analysis based on the electrical connector detection result and generate a local morphology factor; generate a local difference value based on the local morphology factor; and generate the electrical connector detection result based on the local difference value.

[0193] In one embodiment, such as Figure 3 As shown, a computer device is also provided, including a memory and a processor. The memory stores a computer program and an operating system. When the processor executes the computer program, it implements the steps described in the machine vision-based separator sieve surface separation and detection method. The computer device also includes a system bus, internal memory, network structure, display screen, and input devices.

[0194] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps described in the machine vision-based electrical connector detection method.

[0195] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0196] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0197] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0198] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0199] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0200] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0201] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0202] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0203] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0204] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0205] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0206] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0208] One embodiment of this application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above-described methods.

[0209] The computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above description is an example of a computer device and does not constitute a limitation on the computer device. It may include more or fewer components than described above, or a combination of certain components, or different components, such as input / output devices, network access devices, etc.

[0210] The processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0211] In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard drive or RAM. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units of the computer device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0212] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A machine vision-based method for inspecting electrical connectors, characterized in that, The method includes: Establish the triplet template data and template structure topology diagram of the electrical connector, and obtain the binary mask of the electrical connector, specifically including: After disassembling the electrical connector, establish the ternary template data and template structure topology diagram of the electrical connector. The ternary template data includes: structural data, geometric partition data and material property data. An image of the electrical connector is acquired, and edge detection is performed on the electrical connector image to obtain a binary mask of the electrical connector. Generating topology map structure matching values ​​based on the binary mask and template structure topology map of the electrical connector specifically includes: Generate an electrical connector topology map based on the aforementioned electrical connector binarization mask; The electrical connector topology graph and the template structure topology graph are matched using a subgraph isomorphism algorithm to generate a topology graph structure matching value. An electrical connector foreground image is generated based on the binary mask and the electrical connector image. Material matching processing is then performed on the electrical connector foreground image to generate a material property matching degree, specifically including: The binary mask of the electrical connector and the image of the electrical connector are superimposed, and the foreground region is selected to generate a foreground image of the electrical connector. Optical feature analysis is performed on the foreground image of the electrical connector to obtain the material characteristics of the foreground image; Calculate the Euclidean distance between the material features and material property data of the foreground image to generate a material property matching degree; The matching degree of the topology map structure matching value and material property matching degree is calculated, and the electrical connector change image is obtained. The electrical connector detection result is generated based on the electrical connector change image, specifically including: The matching degree of the topology structure matching value and material property matching degree is calculated to generate the comprehensive matching degree of the electrical connector; Based on the overall matching degree of the electrical connector, the triplet template data corresponding to the electrical connector image is obtained; Based on the structural data in the triplet template data, a non-rigid transformation is performed on the foreground image of the electrical connector to obtain a transformed image of the electrical connector. Based on the image of the electrical connector change, perform image morphology analysis and generate electrical connector detection results.

2. The machine vision-based electrical connector inspection method according to claim 1, characterized in that, After disassembling the electrical connector, ternary template data and template structure topology diagram of the electrical connector are established. The ternary template data includes: structural data, geometric partition data, and material property data; including: The physical structure of the electrical connector is disassembled to identify its structural units; The structural units of the electrical connector are constructed using templates according to the pre-defined ternary structure; Each structural unit is assigned an independent structural identifier, and triplet template data and template structural topology diagram are constructed.

3. The machine vision-based electrical connector inspection method according to claim 1, characterized in that, Based on the image of the electrical connector change, perform image morphology analysis and generate electrical connector detection results, including: Based on the electrical connector test results, image morphology analysis is performed, and local morphology factors are generated; Local difference values ​​are generated based on the local morphological factors; The electrical connector test results are generated based on the local difference values.

4. A machine vision-based electrical connector inspection system, characterized in that, The system includes: The binarization mask acquisition module is used to establish the triplet template data and template structure topology diagram of the electrical connector, and to acquire the binarization mask of the electrical connector, specifically including: After disassembling the electrical connector, establish the ternary template data and template structure topology diagram of the electrical connector. The ternary template data includes: structural data, geometric partition data and material property data. An image of the electrical connector is acquired, and edge detection is performed on the electrical connector image to obtain a binary mask of the electrical connector. The structure matching generation module is used to generate topology map structure matching values ​​based on the binary mask and template structure topology map of the electrical connector, specifically including: Generate an electrical connector topology map based on the aforementioned electrical connector binarization mask; The electrical connector topology graph and the template structure topology graph are matched using a subgraph isomorphism algorithm to generate a topology graph structure matching value. The attribute matching generation module is used to generate a foreground image of the electrical connector based on the binary mask of the electrical connector and the image of the electrical connector, and to perform material matching processing on the foreground image of the electrical connector to generate a material attribute matching degree, specifically including: The binary mask of the electrical connector and the image of the electrical connector are superimposed, and the foreground region is selected to generate a foreground image of the electrical connector. Optical feature analysis is performed on the foreground image of the electrical connector to obtain the material characteristics of the foreground image; Calculate the Euclidean distance between the material features and material property data of the foreground image to generate a material property matching degree; The detection result generation module is used to calculate the matching degree of the topology map structure matching value and material property matching degree, obtain the electrical connector change image, and generate the electrical connector detection result based on the electrical connector change image, specifically including: The matching degree of the topology structure matching value and material property matching degree is calculated to generate the comprehensive matching degree of the electrical connector; Based on the overall matching degree of the electrical connector, the triplet template data corresponding to the electrical connector image is obtained; Based on the structural data in the triplet template data, a non-rigid transformation is performed on the foreground image of the electrical connector to obtain a transformed image of the electrical connector. Based on the image of the electrical connector change, perform image morphology analysis and generate electrical connector detection results.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images