An intelligent pin inserting method for an electrical connector based on visual recognition and path optimization

Abstract

The application discloses an intelligent pin inserting method for an electric connector based on visual identification and path optimization, and specifically comprises the following steps: firstly, image acquisition and preprocessing are performed, and pin body identification and classification are performed; then, locating piece hole identification and positioning are performed, and image coordinates of the pin and the hole are converted into world coordinates; subsequently, pin and hole matching optimization is performed, an improved Hungarian algorithm is used to solve minimum total distance matching, a traveling salesman problem is modeled as a path planning problem, the number of task points is counted, and a task point number threshold is set; when the number of task points is less than or equal to the task point number threshold, a dynamic programming algorithm is used to solve an optimal path, otherwise, a genetic algorithm is used to solve an approximately optimal path; finally, a suction nozzle is controlled according to the planned path to sequentially complete pin taking, pin moving and pin inserting actions. The application realizes full-automatic operation of electric connector pin inserting operation, improves pin inserting precision, operation efficiency and system robustness, reduces production cost and improves product yield.

Description

Technical Field

[0001] This invention relates to the field of automated assembly technology for electrical connectors, and in particular to a smart pin method for electrical connectors based on visual recognition and path optimization. Background Technology

[0002] Electrical connectors are indispensable components in electronic devices and are widely used in aerospace, automotive electronics, communication equipment, and consumer electronics. As electronic devices develop towards miniaturization, high density, and high reliability, the number of pins on electrical connectors is increasing, and the pin diameter is becoming smaller, placing extremely high demands on the precision and efficiency of pin assembly.

[0003] Traditional electrical connector pin insertion operations are mostly carried out manually or through semi-automated mechanical methods. Manual operation relies on the operator's visual judgment and fine motor skills, which has the following significant drawbacks:

[0004] (1) Inefficient: The manual picking and insertion of needles is slow and difficult to meet the needs of large-scale production.

[0005] (2) Insufficient precision: The human eye is prone to fatigue under high-intensity work, which leads to inaccurate positioning, and problems such as misaligned insertion, incorrect insertion, and needle damage are likely to occur, resulting in a low yield.

[0006] (3) Poor consistency: The quality of operation by different operators or the same operator at different times fluctuates greatly, making it difficult to guarantee product consistency.

[0007] To address these issues, some existing technologies have introduced automated equipment, such as vibratory feeders and simple mechanical clamps for needle insertion. However, these solutions typically cannot effectively handle situations with highly random incoming material postures (e.g., uncertain needle positions and angles in flexible vibratory feeders) and small, diverse positioning hole sizes, leading to high pick-up failure rates and misinsertion rates. Furthermore, in parallel operations with multiple needles and holes, existing systems often employ fixed sequences or simple nearest-neighbor principles for task planning, failing to optimize the nozzle's movement path and needle-hole matching relationships from a global perspective. This results in redundant work paths, limited efficiency improvements, and difficulty in handling constraints related to needle-hole type mismatches.

[0008] Therefore, how to achieve full automation of the electrical connector pin insertion process, and on this basis, achieve accurate identification and classification of randomly oriented pins, accurate positioning of multiple types of holes, globally optimal pin-hole matching, and efficient motion path planning, are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0009] The purpose of this invention is to provide a smart pin method for electrical connectors based on visual recognition and path optimization, which features high recognition accuracy, excellent matching efficiency, short pin path, and high degree of automation.

[0010] The technical solution for achieving the objective of this invention is: a smart pin method for electrical connectors based on visual recognition and path optimization, comprising the following steps:

[0011] Step 1: Image acquisition and preprocessing. Acquire images of the needles and positioning components in the flexible vibrating plate using an industrial camera, and perform image preprocessing.

[0012] Step 2: Needle identification and classification. The edge of the image is obtained by edge extraction algorithm, and then the contour is detected based on the edge information to identify the needles in the image. The identified needles are then classified by template matching algorithm.

[0013] Step 3: Identification and positioning of positioning hole. The Hough circle detection and contour analysis method is used to detect holes in the positioning component and classify them according to the hole type.

[0014] Step 4: Coordinate system transformation, converting the image coordinates of the needle and hole to world coordinates;

[0015] Step 5: Needle and hole matching optimization. Construct a weighted bipartite graph for needle and hole matching, and use an improved Hungarian algorithm to solve for the minimum total distance matching.

[0016] Step 6: Model the path planning problem as a traveling salesman problem, count the number of task points, and set a threshold for the number of task points to select the path planning algorithm.

[0017] When the number of task points is less than or equal to the task point number threshold, proceed to step 7.

[0018] When the number of task points exceeds the task point threshold, proceed to step 8.

[0019] Step 7: Use dynamic programming to find the optimal path and then proceed to step 9;

[0020] Step 8: Use a genetic algorithm to find the approximate optimal path, and then proceed to step 9;

[0021] Step 9: Motion control and needle insertion execution. Control the nozzle to complete the needle picking, needle moving, and needle insertion actions in sequence according to the planned path.

[0022] Compared with the prior art, the present invention has the following significant advantages: (1) It constructs a complete closed loop of "visual perception-intelligent decision-motion control" technology. Through multi-layer image preprocessing, edge detection and contour analysis, it realizes high-precision identification and classification of random posture needles in flexible vibrating disks. Combined with Hough circle detection and clustering algorithm, it completes the accurate positioning of multiple types of holes in the positioning parts, providing a precise perception basis for subsequent needle insertion operations; (2) In the needle and hole matching stage, a weighted bipartite graph is constructed and an improved Hungarian algorithm is introduced. By setting type constraint penalty terms in the cost matrix, the global total movement distance is solved under the premise of ensuring strict matching of needle and hole types. The minimum pairing scheme not only avoids the problem of misinsertion, but also optimizes the stroke layout of the nozzle as a whole; (3) In the path planning stage, the pin insertion sequence optimization is modeled as a traveling salesman problem, and the solution algorithm is dynamically selected according to the number of task points—dynamic programming is used to ensure global optimality when there are few task points, and genetic algorithm is used to quickly approximate the optimal solution when there are many task points, thus achieving an adaptive balance between work efficiency and solution quality, and shortening the idle stroke time of the nozzle; (4) The fully automated operation of the electrical connector pin insertion is realized. Through intelligent sensing and global optimization technology, the pin insertion accuracy, work efficiency and system robustness are improved, production costs are reduced and product yield is improved. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the intelligent pin method for electrical connectors based on visual recognition and path optimization according to the present invention.

[0024] Figure 2 This is a schematic diagram of the needle identification and classification process in this invention.

[0025] Figure 3 This is a schematic diagram of the process for identifying and locating the positioning hole in this invention.

[0026] Figure 4 This is a flowchart illustrating the path planning algorithm in this invention.

[0027] Figure 5 These are the original images of the needle body and positioning element in the embodiments of the present invention.

[0028] Figure 6 These are images of the needle body and positioning element after preprocessing in an embodiment of the present invention.

[0029] Figure 7 This is a schematic diagram illustrating the needle identification and classification effect in an embodiment of the present invention.

[0030] Figure 8 This is a schematic diagram illustrating the identification and positioning effect of the positioning hole in an embodiment of the present invention.

[0031] Figure 9 This is a schematic diagram of the path planning results in an embodiment of the present invention. Detailed Implementation

[0032] It is readily understood that, based on the technical solution of this invention, various embodiments of the invention can be conceived by those skilled in the art without altering its essential spirit. Therefore, the following detailed descriptions and accompanying drawings are merely illustrative of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on its technical solution. The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Combination Figure 1 The present invention discloses a smart pin method for electrical connectors based on visual recognition and path optimization, characterized by comprising the following steps:

[0034] Step 1: Image acquisition and preprocessing. Acquire images of the needles and positioning components in the flexible vibrating plate using an industrial camera, and perform image preprocessing.

[0035] Step 2: Needle identification and classification. The edge of the image is obtained by edge extraction algorithm, and then the contour is detected based on the edge information to identify the needles in the image. The identified needles are then classified by template matching algorithm.

[0036] Step 3: Identification and positioning of positioning hole. The Hough circle detection and contour analysis method is used to detect holes in the positioning component and classify them according to the hole type.

[0037] Step 4: Coordinate system transformation, converting the image coordinates of the needle and hole to world coordinates;

[0038] Step 5: Needle and hole matching optimization. Construct a weighted bipartite graph for needle and hole matching, and use an improved Hungarian algorithm to solve for the minimum total distance matching.

[0039] Step 6: Model the path planning problem as a traveling salesman problem, count the number of task points, and set a threshold for the number of task points to select the path planning algorithm.

[0040] When the number of task points is less than or equal to the task point number threshold, proceed to step 7.

[0041] When the number of task points exceeds the task point threshold, proceed to step 8.

[0042] Step 7: Use dynamic programming to find the optimal path and then proceed to step 9;

[0043] Step 8: Use a genetic algorithm to find the approximate optimal path, and then proceed to step 9;

[0044] Step 9: Motion control and needle insertion execution. Control the nozzle to complete the needle picking, needle moving, and needle insertion actions in sequence according to the planned path.

[0045] As a specific example, the image acquisition and preprocessing described in step 1 involves acquiring images of the needles and positioning components in the flexible vibrating plate using an industrial camera, and then performing image preprocessing as follows:

[0046] Step 1.1: Binarize the acquired image using the following formula:

[0047]

[0048] in, and Represent the source image and target image in coordinates respectively. The pixel grayscale value at that location; This indicates a preset global threshold. This indicates the value that needs to be assigned when the pixel value is below the threshold.

[0049] Step 1.2: Smooth the binarized image using the median filtering algorithm. The formula is as follows:

[0050]

[0051] in, , Indicates A neighborhood window centered on the center; and These are the source image and the target image in coordinates. The pixel grayscale value at that location; The size of the filter window is indicated by ; This represents the operation of retrieving the median of all values ​​in a set.

[0052] As a specific example, the needle identification and classification in step 2 involves obtaining image edges through an edge extraction algorithm, then performing contour detection based on the edge information to identify the needles in the image, and finally classifying the identified needles using a template matching algorithm, as detailed below:

[0053] Step 2.1: Perform edge detection on the preprocessed needle image to detect all potential contours, as follows:

[0054] Step 2.1.1: Calculate the smoothed image using the Sobel operator. In the horizontal direction and vertical direction The gradient of , where the Sobel operator is expressed as:

[0055]

[0056] For each pixel gradient strength and direction The calculation method is as follows:

[0057]

[0058]

[0059] In the formula Represents pixels gradient strength, Represents pixels The gradient direction is obtained. Approximately horizontal, vertical, and two diagonal directions;

[0060] Step 2.1.2: Perform non-maximum suppression on the gradient intensity map. If the gradient intensity of the current pixel is not a local maximum in its gradient direction, then perform zero-based suppression to refine the edges. The judgment method is as follows:

[0061]

[0062] In the formula This is the output after non-maximum suppression. and for The intensity of two neighboring pixels along the gradient direction;

[0063] Step 2.1.3: Finally determine the edge through dual threshold detection, and determine the dual threshold. and If the gradient value of a certain pixel is greater than or gradient value at and If there is a boundary between them, then treat them as boundaries; if the gradient value of a certain pixel is less than... or gradient value at and If there is no boundary between them, then they are not treated as boundaries;

[0064] Step 2.2: Irregular contour filtering. For the overlapping needle contours that appeared in Step 2.1, a filtering mechanism is set up to remove them, as follows:

[0065] Step 2.2.1: Set the maximum width threshold for the outline. If the outline width is greater than If it is an irregular contour, it will be filtered out.

[0066] Step 2.2.2: Convexity filtering. Traverse each contour, calculate the convexity and make a judgment. The calculation formula is:

[0067]

[0068] In the formula For convexity, The area of ​​the contour. The area of ​​the smallest convex polygon containing all contour points, if If it is, then it is determined to be a normal needle outline. If it is, then it is determined to be an abnormal contour. The preset convexity threshold is used;

[0069] Step 2.3: Identify the needle tail and perform a safe zone detection. The nozzle picks up the needle tail. Since the tail is thinner than the head, and the nozzle has a set width, a safe zone detection is needed to ensure that there are no other needles around the needle tail, and to filter out needles with other needles around the needle tail, as follows:

[0070] Step 2.3.1: Identify the needle tail and obtain the standard contour image. Divide it into two equal halves along the height direction. With the axis pointing downwards as positive, calculate the total number of pixels in both the upper and lower parts separately using the following formula:

[0071]

[0072] Compare and The size of the needle is determined by the side with the smaller value;

[0073] Step 2.3.2: Return to calculate the coordinates of the needle tail. The major axis of the rotated rectangle is known to be... The center is Set from the center along the major axis move Once the head is reached, the tail is in the opposite direction. The coordinates are:

[0074]

[0075]

[0076] In the formula This is the empirical offset. That is, the coordinates of the needle tail;

[0077] Step 2.3.3: Safe zone detection, extracting a circular mask area centered on the needle tail coordinates and with the nozzle diameter as its diameter. Then extract the mask containing the contours of all the remaining needles. ,judge If the value is not empty, it is considered an "unsafe area" and the identified needles are filtered out; otherwise, it is considered a safe area.

[0078] Step 2.4, Needle Type Classification: Each standardized needle outline image is matched with a standardized template for each needle type to identify the needle type, as detailed below:

[0079] Step 2.4.1: Standardize the outline of each needle so that the needle tip faces upward and the needle tail faces downward, that is, around the center of the outline. Rotate the image The degree is determined by an affine transformation, which maps points in the original image to corresponding points in the rotated image using a rotation matrix. The formula is:

[0080]

[0081] Rotation matrix Defined as:

[0082]

[0083] Initial translation Make the center of rotation Then, the translation is adjusted to map the original center point to the new image center, using the following formula:

[0084]

[0085] Rotated image Obtained from the original image through affine transformation:

[0086]

[0087] Step 2.4.2: Perform template matching between the standardized needle outline image obtained in step 2.4.1 and the standardized template of each needle type, and obtain a matching score. The needle type with the highest score is determined. The obtained score cannot be lower than the preset value. If it is lower than the preset value, it is determined to be an unknown type.

[0088] During matching, the length and width of the standardized needle outline image are first subtracted from the length and width of the template. If the absolute value of the difference is greater than the preset standard value, the matching score is directly determined to be 0, i.e., no match. If the difference between the image and the template length and width meets the standard, the standardized image is then matched with a pre-stored standard template library using normalized cross-correlation matching. The matching formula is as follows:

[0089]

[0090] In the formula, Indicates the output result graph Middle position The higher the correlation coefficient value, the stronger the correlation between the template and the template. The higher the matching degree of the image region with the top left corner point; The template pixel value after removing the mean is calculated using the following formula:

[0091]

[0092] The formula for calculating the pixel values ​​of an image region after removing the mean is:

[0093]

[0094] In the formula These represent the width and height of the template image, respectively.

[0095] Step 2.5: Output the needle set. Pack the final result and output it to the computer cache. The output format is: (needle type, needle tail coordinates).

[0096] As a specific example, the hole identification and positioning of the positioning component described in step 3 uses the Hough circle detection and contour analysis method to detect holes in the positioning component and classifies them according to the hole type, as follows:

[0097] Step 3.1: Flood fill the binarized image of the positioning part area. Using the image boundary points as seed points, set the connected background area to zero and retain only the isolated hole area to generate a hole mask image. Then, use edge detection to find the outline and position of the hole.

[0098] Step 3.2: Geometric feature filtering, distinguish between real holes and irregular noise, extract the connected component contours in the hole mask, calculate the area, perimeter and equivalent circle diameter of each contour, filter non-circular noise based on the roundness formula, filter excessively large areas and tiny noise based on the diameter range, and generate an effective hole dataset.

[0099] Step 3.2.1: Calculate roundness using the following formula:

[0100]

[0101] In the formula, This is the roundness value. For the outline area, The perimeter of the outline is used. If the roundness value is greater than the preset roundness threshold, it is determined to be a circular hole; otherwise, it is determined to be an irregular noise point.

[0102] Step 3.2.2: Calculate the physical diameter using the following formula:

[0103]

[0104] In the formula To obtain camera parameters through calibration, representing the number of pixels per millimeter in the image, a maximum and minimum diameter threshold are set. If the outline diameter is greater than the maximum threshold or less than the minimum threshold, it is counted as irregular noise.

[0105] Step 3.3: Aperture Classification. Due to the small differences in aperture diameter and the existence of imaging drift, an unsupervised clustering algorithm is used to cluster aperture diameters. A mapping relationship with standard aperture types is established by ranking the cluster centers, as detailed below:

[0106] Step 3.3.1: Initialize the cluster center according to the preset number of hole types. Random selection One sample, i.e., the diameter of the hole profile, is used as the initial cluster center. ;

[0107] Step 3.3.2: Assign samples to the nearest cluster. For each sample... Calculate its relationship with each cluster center The Euclidean distance is used to assign the cluster to the nearest neighbor, and the assignment results are expressed using a binary indicator variable. express:

[0108]

[0109] Step 3.3.3: Update cluster centers and recalculate the center of each cluster, which is the mean of all samples in that cluster:

[0110]

[0111] Step 3.3.4: Iterative convergence. Repeat steps 3.3.2 and 3.3.3 until the change in cluster centers is less than the threshold, or the preset maximum number of iterations is reached. The formula is:

[0112]

[0113] Step 3.4: Output the hole set. Pack the final result and output it to the computer cache. The output format is: (hole type, hole coordinates).

[0114] As a specific example, the coordinate system transformation described in step 4 converts the image coordinates of the needle and the hole into world coordinates, as follows:

[0115] Step 4.1: Solve for the intrinsic parameter matrix through camera calibration. distortion coefficients and depth information Obtain pixel coordinate system With camera coordinate system The relationship between them, using Zhang Zhengyou's calibration method, is as follows:

[0116]

[0117] Step 4.2: Obtain the camera coordinate system through hand-eye calibration. With the three-axis nozzle coordinate system The relationship between them is transformed using an eye-outside-hand orientation method, resulting in:

[0118]

[0119] In the formula Let the coordinates of the suction nozzle be in the base coordinate system. Let be the coordinates of the suction nozzle in the camera coordinate system; therefore, the transformation relationship is:

[0120]

[0121] As a specific example, the pin-and-hole matching optimization described in step 5 constructs a weighted bipartite graph for pin-and-hole matching and uses an improved Hungarian algorithm to solve for the minimum total distance matching, as detailed below:

[0122] Step 5.1: Construct a weighted bipartite graph, with the left node being the set of needles. ,in The right node is the set of holes. ,in ;

[0123] Step 5.2: Construct the bipartite graph cost matrix ,element Indicates that the needle Insertion hole The cost matrix is ​​improved to:

[0124]

[0125] in M is an infinite penalty value used to enforce type constraints. By setting this value, the algorithm can automatically exclude type mismatches mathematically, eliminating the need for pre-grouping and increasing the algorithm's versatility.

[0126] Step 5.3: Use the improved Hungarian algorithm to solve for the minimum total distance matching. The objective function is to minimize the total cost. Decision variables are introduced. ,when Hour hand Hole Then the objective function is:

[0127]

[0128] The constraints are:

[0129]

[0130] Step 5.4: Output the set of matching pairs. Pack the final results and output them to the computer cache. The output format is: (pin ID, hole ID).

[0131] As a specific example, step 6 involves modeling the path planning problem as a traveling salesman problem, counting the number of task points, and setting a threshold for the number of task points to select a path planning algorithm, as detailed below:

[0132] Each pin-hole matching pair is considered as a "task point". The distance between task points is the distance from the pin insertion position of one hole to the picking position of the next pin. Since the dynamic programming algorithm can obtain the optimal solution, but the time complexity increases exponentially with the number of task points, the dynamic programming algorithm is used when the number of task points is not greater than the threshold; when the number of task points is greater than the threshold, the genetic algorithm is used.

[0133] As a specific example, step 7, which describes using dynamic programming to find the optimal path, is as follows:

[0134] Step 7.1: State Definition, define the dynamic programming state function. This means: starting from point 0, passing through set All points in the, and the final destination point. The minimum cumulative cost;

[0135] Step 7.2: Construct the state transition equation. The initial state has a set size of 2, meaning the starting point is directly connected to a certain point. The initial state equation is:

[0136]

[0137] The state equation for the recursive state is:

[0138]

[0139] in Indicates from set Remove point The subset that follows, while also recording the optimal predecessor node. Used for path reconstruction;

[0140] Step 7.3: Traverse all possible endpoints Calculate the total cost of returning to the starting point 0, and obtain the minimum cost:

[0141]

[0142] Step 7.4, based on the record's predecessor pointer Perform path reconstruction:

[0143]

[0144] As a specific example, step 8, which describes using a genetic algorithm to find the approximate optimal path, is as follows:

[0145] Step 8.1: Chromosome encoding. The population size is preset. Each individual is encoded as a sequence of task points. The set of task points is... Starting from 0, each chromosome individual is represented as a sequence of task points: The complete path is: ;

[0146] Step 8.2: Using a tournament selection strategy, randomly select K individuals from the population, and choose the individual with the lowest cost to enter the next generation:

[0147]

[0148] Step 8.3: Perform partial mapping crossover, randomly select two crossover points, exchange the gene segments of the parent individuals in that interval, and eliminate gene conflicts in the offspring individuals through the mapping relationship to ensure that each task point appears only once;

[0149] Set the parent individual as Randomly select the intersection interval For each position within the interval ,exchange The value at the position;

[0150] Step 8.4: Perform fragment inversion mutation operation: Randomly select sequence fragments with a preset mutation probability and reverse their order, randomly selecting two positions. ,and Reverse the intermediate sequence:

[0151]

[0152] Step 8.5: Introduce an elite retention mechanism, retaining the globally optimal individual in each iteration until a preset number of iterations is reached. The globally optimal individual in each generation is:

[0153]

[0154] As a specific example, the motion control and needle insertion execution described in step 9 controls the nozzle to sequentially complete the needle picking, needle moving, and needle insertion actions according to the planned path, as follows:

[0155] Step 9.1, coordinate transformation and motion command generation: Based on the optimal access sequence output by the path planning algorithm, the image pixel coordinates of the target point are first transformed to the physical coordinates in the robot's base coordinate system, i.e., world coordinates, using the transformation matrix obtained from camera calibration and hand-eye calibration.

[0156] Step 9.2: After receiving the command, the motion controller drives the X, Y, and Z axes of the three-axis servo motor to move along the planned path. During the movement, S-curve acceleration and deceleration or trapezoidal speed planning is used to ensure smooth start and stop of the suction nozzle and avoid needle body displacement or damage due to impact. When the suction nozzle reaches the first target point, i.e. the tail of the needle, the controller triggers the vacuum solenoid valve to open, and the needle is sucked and fixed by the negative pressure at the end of the suction nozzle. At the same time, the vacuum degree is detected by the pressure sensor to confirm successful suction. Subsequently, the suction nozzle carries the needle body and moves along the path to the corresponding hole. At this time, the Z axis descends and inserts the needle into the hole. During the insertion process, the insertion depth is monitored in real time by force feedback or position encoder to prevent overpressure.

[0157] Step 9.3: After completing one needle insertion, the controller closes the vacuum valve and releases the needle. Then the suction nozzle rises and moves to the next target point, repeating the suction and insertion action until all matching needle pairs have been processed. Throughout the entire process, the control system monitors the position, speed and I / O status of each axis in real time and feeds back the execution results to the host computer. If an abnormality occurs, the system will automatically pause and alarm, or retry according to the preset strategy.

[0158] Example

[0159] This embodiment selects a set of electrical connectors from actual production, containing three different types of pins with diameters of 0.5mm, 0.6mm, and 1.0mm, and corresponding hole positions, for automatic pin insertion simulation and experimentation. The experimental environment is based on an industrial vision platform, using a computer with an i7-14650 CPU and an RTX4060 GPU for data processing.

[0160] Combination Figures 1-4 This embodiment provides a smart connector pin method based on visual recognition and path optimization, with the following specific steps:

[0161] Step 1: Acquire raw images of the flexible vibratory feeder and positioning components using an industrial camera, such as... Figure 5 As shown, using the binarization process described in step 1.1, a preset global threshold is set. The value is 158, so the image is converted to a binary image; then, the median filtering algorithm in step 1.2 is used, and a 3×3 filtering window is selected for smoothing, effectively removing noise and obtaining a clear preprocessed image as shown. Figure 6 As shown.

[0162] Step 2: Perform Sobel edge detection and non-maximum suppression on the preprocessed needle image. In the irregular contour filtering stage, a preset convexity threshold is set. A value of 0.6 is used to eliminate overlapping or incomplete needle outlines; during safe zone inspection, the needle tail coordinates are identified based on the nozzle diameter. Finally, needle type classification is performed. The identified needles are matched with a standard template library using normalized cross-correlation matching. A preset score threshold of 0.65 is set to accurately classify the needles into three preset types. The visual effects of recognition and classification are as follows: Figure 7 As shown in the result image, the method accurately identifies all needles and their types in the image, and marks and filters out overlapping needles and needle bodies with other needles around the needle tail.

[0163] Step 3: Flood fill the positioning area to generate a hole mask. In the geometric feature selection, set a preset roundness threshold. The accuracy is set to 0.7, and noise is filtered based on the physical diameter range. For the three types of holes, an unsupervised clustering algorithm is used, with an initial cluster center count K=3 and a maximum iteration count of 60. Holes are automatically classified using the cluster centers after convergence, outputting a hole set in the format: (hole type, hole coordinates). The recognition effect is as follows: Figure 8 As shown in the result image, the method accurately identified and classified all the holes on the positioning component. Thanks to the use of unsupervised clustering algorithm, it accurately distinguished between holes with very small differences of 0.5 mm and 0.6 mm.

[0164] Step 4: Use Zhang Zhengyou's calibration method to obtain the camera intrinsic parameters. Combine the calibration matrix with the eye outside the hand to uniformly transform the pin tail pixel coordinates and hole position pixel coordinates obtained in Steps 2 and 3 to world coordinates in the robot's base coordinate system.

[0165] Step 5: Construct a weighted bipartite graph of the needle set and the hole set. In the cost matrix, set an infinite penalty value M for needle-hole pairs that do not match the type, and use the improved Hungarian algorithm to solve for the set of matching pairs (needle ID, hole ID) with the minimum total distance.

[0166] Step 6: Model the matched pin insertion task as a TSP problem. In this embodiment, due to the large number of task points exceeding the preset threshold, the system automatically selects the genetic algorithm described in Step 8 for path planning. The preset population size of the genetic algorithm is set to 60, the number of iterations is 800, and the mutation probability is 0.25. Through tournament selection, partial mapping crossover, and fragment inversion mutation, and by introducing an elite retention mechanism, the system finally outputs a globally approximately optimal pin insertion path. The logical diagram and results of the path planning are shown below. Figure 9 As shown in the figure, the pin-hole matching pairs are represented by gray dashed lines, and the shortest movement trajectory of the nozzle is represented by red arrows.

[0167] Step 9: The motion controller receives the optimal access sequence command and drives the three-axis servo motor to move the suction nozzle. The suction nozzle sequentially picks up needles in the safe zone of the vibratory feeder according to the planned order and moves them precisely to the corresponding holes to complete the insertion action. During execution, the S-curve speed planning is used to ensure smoothness until all three types of needles are accurately inserted into their corresponding holes.

[0168] Experimental results show that by setting reasonable thresholds, such as threshold 158, convexity 0.6, and roundness 0.7, as well as efficient path planning parameters, such as 800 iterations and a variation rate of 0.25, the present invention significantly improves the automation level and operating efficiency of electrical connector pins and effectively avoids misinsertion and operational conflicts.

[0169] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent connector pins based on visual recognition and path optimization, characterized in that, Includes the following steps: Step 1: Image acquisition and preprocessing. Acquire images of the needles and positioning components in the flexible vibrating plate using an industrial camera, and perform image preprocessing. Step 2: Needle identification and classification. The edge extraction algorithm is used to obtain the image edge, and then the contour detection is performed based on the edge information to identify the needle in the image. The template matching algorithm is used to classify the identified needle. Step 3: Identification and positioning of positioning hole. The holes in the positioning component are detected using the Hough circle detection and contour analysis method, and classified according to the hole type. Step 4: Coordinate system transformation, converting the image coordinates of the needle and hole to world coordinates; Step 5: Needle and hole matching optimization. Construct a weighted bipartite graph for needle and hole matching, and use the improved Hungarian algorithm to solve for the minimum total distance matching. Step 6: Model the path planning problem as a traveling salesman problem, count the number of task points, and set a threshold for the number of task points to select the path planning algorithm. When the number of task points is less than or equal to the task point number threshold, proceed to step 7. When the number of task points exceeds the task point threshold, proceed to step 8. Step 7: Use dynamic programming to find the optimal path and then proceed to step 9; Step 8: Use a genetic algorithm to find the approximate optimal path, and then proceed to step 9; Step 9: Motion control and needle insertion execution. Control the nozzle to complete the needle picking, needle moving, and needle insertion actions in sequence according to the planned path.

2. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, The image acquisition and preprocessing described in step 1 involves acquiring images of the needles and positioning components in the flexible vibrating plate using an industrial camera, and then performing image preprocessing as follows: Step 1.1: Binarize the acquired image using the following formula: in, and Represent the source image and target image in coordinates respectively. The pixel grayscale value at that location; This indicates a preset global threshold. This indicates the value that needs to be assigned when the pixel value is below the threshold. Step 1.2: Smooth the binarized image using the median filtering algorithm. The formula is as follows: in, , Indicates A neighborhood window centered on the center; and These are the source image and the target image in coordinates. The pixel grayscale value at that location; The size of the filter window is indicated by ; This represents the operation of retrieving the median of all values ​​in a set.

3. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 2, needle identification and classification, involves obtaining image edges using an edge extraction algorithm, performing contour detection based on the edge information to identify needles in the image, and then classifying the identified needles using a template matching algorithm, as detailed below: Step 2.1: Perform edge detection on the preprocessed needle image to detect all potential contours, as follows: Step 2.1.1: Calculate the smoothed image using the Sobel operator. In the horizontal direction and vertical direction The gradient of , where the Sobel operator is expressed as: For each pixel gradient strength and direction The calculation method is as follows: In the formula Represents pixels gradient strength, Represents pixels The gradient direction is obtained. Approximately horizontal, vertical, and two diagonal directions; Step 2.1.2: Perform non-maximum suppression on the gradient intensity map. If the gradient intensity of the current pixel is not a local maximum in its gradient direction, then perform zero-based suppression to refine the edges. The judgment method is as follows: In the formula This is the output after non-maximum suppression. and for The intensity of two neighboring pixels along the gradient direction; Step 2.1.3: Finally determine the edge through dual threshold detection, and determine the dual threshold. and If the gradient value of a certain pixel is greater than or gradient value at and If there is a boundary between them, then treat them as boundaries; if the gradient value of a certain pixel is less than... or gradient value at and If there is no boundary between them, then they are not treated as boundaries; Step 2.2: Irregular contour filtering. For the overlapping needle contours that appeared in Step 2.1, a filtering mechanism is set up to remove them, as follows: Step 2.2.1: Set the maximum width threshold for the outline. If the outline width is greater than If it is an irregular contour, it will be filtered out. Step 2.2.2: Convexity filtering. Traverse each contour, calculate the convexity and make a judgment. The calculation formula is: In the formula For convexity, The area of ​​the contour. The area of ​​the smallest convex polygon containing all contour points, if If it is, then it is determined to be a normal needle outline. If it is, then it is determined to be an abnormal contour. The preset convexity threshold is used; Step 2.3: Identify the needle tail and perform a safe zone detection. The nozzle picks up the needle tail. Since the tail is thinner than the head, and the nozzle has a set width, a safe zone detection is needed to ensure that there are no other needles around the needle tail, and to filter out needles with other needles around the needle tail, as follows: Step 2.3.1: Identify the needle tail and obtain the standard contour image. Divide it into two equal halves along the height direction. With the axis pointing downwards as positive, calculate the total number of pixels in both the upper and lower parts separately using the following formula: Compare and The size of the needle is determined by the side with the smaller value; Step 2.3.2: Return to calculate the coordinates of the needle tail. The major axis of the rotated rectangle is known to be... The center is Set from the center along the major axis move Once the head is reached, the tail is in the opposite direction. The coordinates are: In the formula This is the empirical offset. That is, the coordinates of the needle tail; Step 2.3.3: Safe zone detection, extracting a circular mask area centered on the needle tail coordinates and with the nozzle diameter as its diameter. Then extract the mask containing the contours of all the remaining needles. ,judge If the value is not empty, it is considered an "unsafe area" and the identified needles are filtered out; otherwise, it is considered a safe area. Step 2.4, Needle Type Classification: Each standardized needle outline image is matched with a standardized template for each needle type to identify the needle type, as detailed below: Step 2.4.1: Standardize the outline of each needle so that the needle tip faces upward and the needle tail faces downward, that is, around the center of the outline. Rotate the image The degree is calculated using an affine transformation, which maps points in the original image to corresponding points in the rotated image using a rotation matrix. The formula is: Rotation matrix Defined as: Initial translation Make the center of rotation Then, the translation is adjusted to map the original center point to the new image center, using the following formula: Rotated image Obtained from the original image through affine transformation: Step 2.4.2: Perform template matching between the standardized needle outline image obtained in step 2.4.1 and the standardized template of each needle type, and obtain a matching score. The needle type with the highest score is determined. The obtained score cannot be lower than the preset value. If it is lower than the preset value, it is determined to be an unknown type. During matching, the length and width of the standardized needle outline image are first subtracted from the length and width of the template. If the absolute value of the difference is greater than the preset standard value, the matching score is directly determined to be 0, i.e., no match. If the difference between the image and the template length and width meets the standard, the standardized image is then matched with a pre-stored standard template library using normalized cross-correlation matching. The matching formula is as follows: In the formula, Indicates the output result graph Middle position The higher the correlation coefficient value, the stronger the correlation between the template and the template. The higher the matching degree of the image region with the top left corner point; The template pixel value after removing the mean is calculated using the following formula: The formula for calculating the pixel values ​​of an image region after removing the mean is: In the formula These represent the width and height of the template image, respectively. Step 2.5: Output the needle set. Pack the final result and output it to the computer cache. The output format is: (needle type, needle tail coordinates).

4. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 3, which involves identifying and locating the holes in the positioning components, uses the Hough circle detection and contour analysis method to detect the holes and classifies them according to their shape, as detailed below: Step 3.1: Flood fill the binarized image of the positioning part area. Using the image boundary points as seed points, set the connected background area to zero and retain only the isolated hole area to generate a hole mask image. Then, use edge detection to find the outline and position of the hole. Step 3.2: Geometric feature filtering, distinguish between real holes and irregular noise, extract the connected component contours in the hole mask, calculate the area, perimeter and equivalent circle diameter of each contour, filter non-circular noise based on the roundness formula, filter excessively large areas and tiny noise based on the diameter range, and generate an effective hole dataset. Step 3.2.1: Calculate roundness using the following formula: In the formula, This is the roundness value. For the outline area, The perimeter of the outline is used. If the roundness value is greater than the preset roundness threshold, it is determined to be a circular hole; otherwise, it is determined to be an irregular noise point. Step 3.2.2: Calculate the physical diameter using the following formula: In the formula To obtain camera parameters through calibration, representing the number of pixels per millimeter in the image, a maximum and minimum diameter threshold are set. If the outline diameter is greater than the maximum threshold or less than the minimum threshold, it is counted as irregular noise. Step 3.3: Aperture Classification. Due to the small differences in aperture diameter and the existence of imaging drift, an unsupervised clustering algorithm is used to cluster aperture diameters. A mapping relationship with standard aperture types is established by ranking the cluster centers, as detailed below: Step 3.3.1: Initialize the cluster center according to the preset number of hole types. Random selection One sample, i.e., the diameter of the hole profile, is used as the initial cluster center. ; Step 3.3.2: Assign samples to the nearest cluster. For each sample... Calculate its relationship with each cluster center The Euclidean distance is used to assign the cluster to the nearest neighbor, and the assignment results are expressed using a binary indicator variable. express: Step 3.3.3: Update cluster centers and recalculate the center of each cluster, which is the mean of all samples in that cluster: Step 3.3.4: Iterative convergence. Repeat steps 3.3.2 and 3.3.3 until the cluster center change is less than the threshold, or the preset maximum number of iterations is reached. The formula is: Step 3.4: Output the hole set. Pack the final result and output it to the computer cache. The output format is: (hole type, hole coordinates).

5. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 4 involves a coordinate system transformation, converting the image coordinates of the needle and the hole into world coordinates, as detailed below: Step 4.1: Solve for the intrinsic parameter matrix through camera calibration. distortion coefficients and depth information Obtain pixel coordinate system With camera coordinate system The relationship between them, using Zhang Zhengyou's calibration method, is as follows: Step 4.2: Obtain the camera coordinate system through hand-eye calibration. With the three-axis nozzle coordinate system The relationship between them is transformed using an eye-outside-hand orientation method, resulting in: In the formula Let the coordinates of the suction nozzle be in the base coordinate system. Let be the coordinates of the suction nozzle in the camera coordinate system; therefore, the transformation relationship is: 。 6. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 5 describes the pin-and-hole matching optimization, which involves constructing a weighted bipartite graph for pin-and-hole matching and using an improved Hungarian algorithm to solve for the minimum total distance matching, as detailed below: Step 5.1: Construct a weighted bipartite graph, with the left node being the set of needles. ,in ; The right node is the set of holes. ,in ; Step 5.2: Construct the bipartite graph cost matrix ,element Indicates that the needle Insertion hole The cost is reduced, and the cost matrix is ​​improved to: in M is an infinite penalty value used to enforce type constraints. By setting this value, the algorithm can automatically exclude type mismatches mathematically, eliminating the need for pre-grouping and increasing the algorithm's versatility. Step 5.3: Use the improved Hungarian algorithm to solve for the minimum total distance matching. The objective function is to minimize the total cost. Decision variables are introduced. ,when Hour hand Hole Then the objective function is: The constraints are: Step 5.4: Output the set of matching pairs. Pack the final results and output them to the computer cache. The output format is: (pin ID, hole ID).

7. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 6 involves modeling the path planning problem as a traveling salesman problem, counting the number of task points, and setting a threshold for the number of task points to select the path planning algorithm. The details are as follows: Each pin-hole matching pair is considered a "task point". The distance between task points is the distance from the pin insertion position of one hole to the picking position of the next pin. Since the dynamic programming algorithm can obtain the optimal solution, but the time complexity increases exponentially with the number of task points, the dynamic programming algorithm is used when the number of task points is not greater than the threshold; when the number of task points is greater than the threshold, the genetic algorithm is used.

8. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 7, which describes using dynamic programming to find the optimal path, is as follows: Step 7.1: State Definition, define the dynamic programming state function. This means: starting from point 0, passing through set All points in the, and the final destination point. The minimum cumulative cost; Step 7.2: Construct the state transition equation. The initial state has a set size of 2, meaning the starting point is directly connected to a certain point. The initial state equation is: The state equation for the recursive state is: in Indicates from set Remove point The subset that follows, while also recording the optimal predecessor node. Used for path reconstruction; Step 7.3: Traverse all possible endpoints Calculate the total cost of returning to the starting point 0, and obtain the minimum cost: Step 7.4, based on the record's predecessor pointer Perform path reconstruction: 。 9. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 8, which describes using a genetic algorithm to find the approximate optimal path, is as follows: Step 8.1: Chromosome encoding. The population size is preset. Each individual is encoded as a sequence of task points. The set of task points is... Starting from 0, each chromosome individual is represented as a sequence of task points: The complete path is: ; Step 8.2: Using a tournament selection strategy, randomly select K individuals from the population, and choose the individual with the lowest cost to enter the next generation: Step 8.3: Perform partial mapping crossover, randomly select two crossover points, exchange the gene segments of the parent individuals in that interval, and eliminate gene conflicts in the offspring individuals through the mapping relationship to ensure that each task point appears only once; Set the parent individual as Randomly select the intersection interval For each position within the interval ,exchange The value at the position; Step 8.4: Perform fragment inversion mutation operation: Randomly select sequence fragments with a preset mutation probability and reverse their order, randomly selecting two positions. ,and Reverse the intermediate sequence: Step 8.5: Introduce an elite retention mechanism, retaining the globally optimal individual in each iteration until a preset number of iterations is reached. The globally optimal individual in each generation is: 。 10. The intelligent connector pin method based on visual recognition and path optimization according to claim 1, characterized in that, Step 9 describes motion control and needle insertion execution, which controls the nozzle to sequentially complete the needle picking, needle moving, and needle insertion actions according to the planned path, as detailed below: Step 9.1, coordinate transformation and motion command generation: Based on the optimal access sequence output by the path planning algorithm, the image pixel coordinates of the target point are first transformed to the physical coordinates in the robot's base coordinate system, i.e., world coordinates, using the transformation matrix obtained from camera calibration and hand-eye calibration. Step 9.2: After receiving the command, the motion controller drives the X, Y, and Z axes of the three-axis servo motor to move along the planned path. During the movement, S-curve acceleration and deceleration or trapezoidal speed planning is used to ensure smooth start and stop of the suction nozzle and avoid needle body displacement or damage due to impact. When the suction nozzle reaches the first target point, i.e. the tail of the needle, the controller triggers the vacuum solenoid valve to open, and the needle is sucked and fixed by the negative pressure at the end of the suction nozzle. At the same time, the vacuum degree is detected by the pressure sensor to confirm successful suction. Subsequently, the suction nozzle carries the needle body and moves along the path to the corresponding hole. At this time, the Z axis descends and inserts the needle into the hole. During the insertion process, the insertion depth is monitored in real time by force feedback or position encoder to prevent overpressure. Step 9.3: After completing one needle insertion, the controller closes the vacuum valve and releases the needle. Then the suction nozzle rises and moves to the next target point, repeating the suction and insertion action until all matching needle pairs have been processed. Throughout the entire process, the control system monitors the position, speed and I / O status of each axis in real time and feeds back the execution results to the host computer. If an abnormality occurs, the system will automatically pause and alarm, or retry according to the preset strategy.

Images