Method and system for precise positioning and placement of circuit board components based on vision guidance

Abstract

This invention provides a method and system for precisely positioning and placing components on a circuit board based on vision guidance. This method involves obtaining multi-angle image information of the component to be placed, determining a three-dimensional point cloud model and depth map sequence, and determining its type information. The method then segments the point cloud model using a region growing algorithm to extract features for pose estimation. The method then uses placement parameters and posture deviation information to control a robotic arm. Finally, the method optimizes motion parameters using an adaptive optical flow algorithm until accuracy requirements are met. This method improves component placement accuracy and efficiency, while reducing placement failure rates.

Description

Technical Field

[0001] The present invention relates to the field of mounting technology, and in particular to a method and system for accurately positioning and mounting components on a circuit board based on vision guidance. Background Art

[0002] As electronic products evolve toward miniaturization and higher integration, the requirements for circuit board component placement accuracy are becoming increasingly stringent. Traditional circuit board component placement technologies, which primarily rely on manual operation or simple mechanical positioning systems, cannot meet the modern electronics manufacturing industry's demand for high-precision, high-efficiency placement. Currently, circuit board component placement is primarily performed using automatic placement machines, which use vision systems to identify and locate components, and then a robotic arm precisely places the components in the designated locations on the circuit board.

[0003] However, the existing circuit board component placement technology still has some defects and shortcomings. First, the traditional two-dimensional visual recognition system has a low recognition accuracy when dealing with components with complex shapes or special materials. In particular, for components with similar appearances but different functions, it is easy to produce misidentification, resulting in placement errors. Secondly, the existing technology mainly relies on a single algorithm for component posture estimation. It lacks adaptability to the posture changes of components during the picking process, and it is difficult to adjust the placement strategy in real time, which affects the final placement accuracy. In addition, the traditional placement system lacks dynamic optimization capabilities. In the process of picking and placing components, the position deviation caused by factors such as mechanical vibration and thermal expansion cannot be effectively compensated, making it difficult to meet the requirements of micron-level placement accuracy.

[0004] With the continuous development of electronic product manufacturing technology, the requirements for circuit board component placement accuracy and efficiency are becoming increasingly higher. There is an urgent need for a placement method that can achieve accurate component identification, real-time posture estimation, and dynamic trajectory optimization to improve placement quality and production efficiency. Summary of the Invention

[0005] The embodiments of the present invention provide a method and system for accurately positioning and mounting components on a circuit board based on vision guidance, which can solve the problems in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for accurately positioning and placing components on a circuit board based on vision guidance, comprising:

[0007] Acquire multi-angle image information of components to be mounted, determine a three-dimensional point cloud model and a depth map sequence of the components to be mounted based on the multi-angle image information, perform three-dimensional feature matching on the components to be mounted according to a preset component feature library, and obtain type information of the components to be mounted;

[0008] The region growing algorithm is used to segment the 3D point cloud model of the components to be mounted, extract the geometric features and surface texture features of the components, and combine it with the iterative closest point algorithm to estimate the pose of the components to be mounted, and obtain the pose deviation information of the components to be mounted relative to the preset mounting reference point;

[0009] Calling preset component placement parameters according to the type information, and controlling the robotic arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information;

[0010] According to the depth map sequence, the three-dimensional motion trajectory of the components is calculated based on the adaptive optical flow algorithm, and the motion parameters of the robotic arm are dynamically optimized until the position and angle of the components to be placed meet the preset placement accuracy requirements; when the position and angle of the components to be placed meet the preset placement accuracy requirements, the robotic arm is controlled to place the components to be placed at the target position on the circuit board.

[0011] Determining a three-dimensional point cloud model and a depth map of a component to be mounted based on the multi-angle image information, performing three-dimensional feature matching on the component to be mounted according to a preset component feature library, and obtaining type information of the component to be mounted includes:

[0012] Obtaining point cloud data of the component to be identified at multiple viewing angles, and extracting local feature descriptors of the point cloud data at each viewing angle, wherein the local feature descriptors include principal curvature eigenvalues, normal vector direction angles, and local curvature radius;

[0013] Calculate the point-to-point feature similarity weight based on the local feature descriptor, substitute the feature similarity weight into the minimization registration error function, solve the rotation matrix and translation vector of the point cloud registration, and align and fuse the point cloud data under multiple perspectives;

[0014] A multi-scale feature histogram is constructed for the fused point cloud data. For each point in the point cloud data, the local geometric features of the point in different scale neighborhoods are extracted. The local geometric features are weighted statistically analyzed based on the Gaussian kernel function to obtain the global feature representation of the point.

[0015] A feature matching network based on the attention mechanism is established, the global feature representation is input into the feature matching network, the feature correspondence score is calculated through query mapping and key-value mapping, a feature matching metric is constructed in combination with regularization constraints, and the feature matching metric is matched with a preset component feature library to obtain the type information of the component to be identified.

[0016] The region growing algorithm is used to segment the 3D point cloud model of the components to be mounted, extract the geometric features and surface texture features of the components, and combine iterative closest point algorithm to estimate the pose of the components. The following steps are involved:

[0017] Calculating the neighborhood covariance matrix eigenvalues ​​of each point in the three-dimensional point cloud data corresponding to the three-dimensional point cloud model, constructing a point cloud curvature index evaluation function based on the eigenvalues, and selecting the point with the local minimum curvature as the seed point for region growth;

[0018] A region growing criterion function is constructed based on the seed point, wherein the region growing criterion function includes a normal vector difference term, a curvature difference term, and a texture feature difference term. The three-dimensional point cloud model is segmented by calculating the normal vector angle, curvature change rate, and texture feature similarity between the point to be processed and the grown region.

[0019] The local texture descriptor is extracted from the segmented point cloud area, and the Gaussian curvature of the point cloud area is calculated as the local geometric descriptor. According to the local texture descriptor and the local geometric descriptor, point pair registration is performed through the iterative nearest point algorithm to obtain the rotation matrix and translation vector of the component to be processed, and the rotation matrix and the translation vector are used as the pose estimation result of the component to be processed.

[0020] Calling preset component placement parameters according to the type information, and controlling the robot arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information includes:

[0021] Extracting corresponding placement reference parameters from a component standard parameter library according to the type information, wherein the placement reference parameters include the spatial coordinates of the target placement position, the range of the allowable placement angle, the limit value of the maximum placement pressure, and the judgment standard of the placement accuracy;

[0022] Determine the current spatial position and attitude angle of the component to be mounted, calculate the coordinate difference between the current spatial position and the target mounting position, compare the attitude angle with the range of the allowable mounting angle, and generate a position compensation vector and an angle correction value;

[0023] planning the motion trajectory of the manipulator according to the position compensation vector and the angle correction value, decomposing the motion trajectory into a horizontal displacement adjustment phase and a vertical posture adjustment phase, and calculating motion parameters of the manipulator joints in the two phases respectively;

[0024] The position error and angle error of the component to be mounted relative to the target mounting position are continuously detected. When the position error and the angle error both meet the judgment criteria of the mounting accuracy, the current motion state of the robot arm is locked and the final mounting action is performed.

[0025] According to the depth map sequence, the three-dimensional motion trajectory of the component is calculated based on the adaptive optical flow algorithm, and the motion parameters of the robot arm are dynamically optimized until the position and angle of the component to be placed meet the preset placement accuracy requirements, including:

[0026] Performing multi-scale decomposition on the depth map sequence to obtain depth feature maps at different resolutions, and calculating spatial position information of components to be mounted based on the depth feature maps;

[0027] Based on the grayscale change relationship between the depth feature maps of adjacent frames, an optical flow displacement calculation rule is established. The optical flow displacement calculation rule includes grayscale consistency constraints and motion smoothness constraints. Adaptive adjustment of local motion estimation is achieved by adjusting the constraint coefficient;

[0028] Combining the spatial position information and the optical flow displacement calculation results, analyzing the depth value differences of corresponding points in the depth feature maps of adjacent frames, determining the depth change weights, and correcting the displacement calculation results according to the depth change weights to obtain the three-dimensional motion parameters of the components to be mounted;

[0029] Constructing a motion trajectory of the component according to the three-dimensional motion parameters, analyzing the acceleration changes at adjacent moments in the motion trajectory, and reducing the acceleration fluctuation of the trajectory curve through iterative optimization to obtain smooth motion trajectory data;

[0030] The smooth motion trajectory data is converted into the robot arm joint motion parameters, the motion state of the robot arm end effector is calculated based on the joint motion parameters, and the position and posture information of the components to be mounted are collected in real time. When the position and posture information meet the preset mounting accuracy requirements, the component mounting operation is executed.

[0031] Combining the spatial position information and the optical flow displacement calculation results, analyzing the depth value differences of corresponding points in the depth feature maps of adjacent frames, determining the depth change weight, and correcting the displacement calculation results according to the depth change weight to obtain the three-dimensional motion parameters of the components to be mounted, including:

[0032] The depth feature map in the previous time series is set as the reference frame, and the depth feature map in the next time series is set as the current frame;

[0033] Detecting feature points in the reference frame, performing descriptor extraction on the feature points, searching for corresponding feature points in the current frame based on the descriptors, establishing feature point correspondences, and calculating spatial position changes of the feature points based on the feature point correspondences;

[0034] An optical flow tracking algorithm is used to calculate the position offset of the feature point in the reference frame in the current frame, and the position offset is converted into a three-dimensional spatial displacement in combination with camera parameters to obtain optical flow motion information of the feature point;

[0035] Extracting depth values ​​of corresponding feature points in the reference frame and the current frame, calculating a depth difference between each pair of feature points, and setting a depth weight based on the depth difference, wherein the depth weight is used to characterize the reliability of the depth measurement of the feature points;

[0036] Performing depth-weighted fusion on the spatial position change and the optical flow motion information, wherein the fusion weight is determined by the depth weight, and obtaining a motion vector of the feature point through weighted fusion;

[0037] The rigid body motion parameters of the component to be mounted are estimated based on the motion vector, the rigid body motion parameters are decomposed into rotation components and translation components, and the three-dimensional motion parameters of the component to be mounted are output.

[0038] A second aspect of an embodiment of the present invention provides a system for accurately positioning and placing components on a circuit board based on vision guidance, comprising:

[0039] The first unit is configured to obtain multi-angle image information of components to be mounted, determine a three-dimensional point cloud model and a depth map sequence of the components to be mounted based on the multi-angle image information, perform three-dimensional feature matching on the components to be mounted according to a preset component feature library, and obtain type information of the components to be mounted;

[0040] The second unit is used to segment the three-dimensional point cloud model of the components to be mounted using the region growing algorithm, extract the geometric features and surface texture features of the components, and estimate the pose of the components in combination with the iterative closest point algorithm to obtain the pose deviation information of the components to be mounted relative to the preset mounting reference point;

[0041] A third unit is configured to call preset component placement parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information;

[0042] The fourth unit is used to calculate the three-dimensional motion trajectory of the components based on the adaptive optical flow algorithm according to the depth map sequence, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the components to be placed meet the preset placement accuracy requirements; when the position and angle of the components to be placed meet the preset placement accuracy requirements, the robotic arm is controlled to place the components to be placed at the target position on the circuit board.

[0043] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0044] processor;

[0045] a memory for storing processor-executable instructions;

[0046] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0047] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0048] The beneficial effects of this application are as follows:

[0049] The vision-guided method for precise positioning and placement of circuit board components provided by the present invention achieves accurate identification and classification of different types of components through multi-angle image acquisition and three-dimensional point cloud model construction, combined with deep learning feature matching technology, effectively solving the problem of difficulty in identifying complex components in traditional placement methods.

[0050] The pose estimation of components is performed by combining the region growing algorithm and the iterative closest point algorithm, which can accurately capture the geometric features and surface texture features of the components, calculate the pose deviation relative to the preset placement reference point, improve the accuracy of component pose adjustment before placement, and reduce the placement failure rate caused by incorrect posture.

[0051] Based on the adaptive optical flow algorithm, the three-dimensional motion trajectory of components is calculated and dynamically optimized in real time, realizing the adaptive adjustment of the robot arm motion parameters, making the placement process more stable and controllable. At the same time, the placement accuracy is guaranteed through the feedback mechanism of the depth map sequence, which significantly improves the efficiency and success rate of circuit board component placement. It is particularly suitable for the precision placement of high-density and small-sized components. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of the process of the method for accurately positioning and placing components on a circuit board based on vision guidance according to an embodiment of the present invention;

[0053] Figure 2 This is a bar chart comparing component recognition performance based on multi-angle three-dimensional point clouds according to an embodiment of the present invention;

[0054] Figure 3 This is a control flow chart of the component precision placement process according to an embodiment of the present invention;

[0055] Figure 4 This is a bar chart comparing the performance of the component three-dimensional motion parameter estimation method based on deep feature fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0057] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0058] Figure 1 FIG. 1 is a flow chart of a method for accurately positioning and mounting components on a circuit board based on vision guidance according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0059] Acquire multi-angle image information of components to be mounted, determine a three-dimensional point cloud model and a depth map sequence of the components to be mounted based on the multi-angle image information, perform three-dimensional feature matching on the components to be mounted according to a preset component feature library, and obtain type information of the components to be mounted;

[0060] The region growing algorithm is used to segment the 3D point cloud model of the components to be mounted, extract the geometric features and surface texture features of the components, and combine it with the iterative closest point algorithm to estimate the pose of the components to be mounted, and obtain the pose deviation information of the components to be mounted relative to the preset mounting reference point;

[0061] Calling preset component placement parameters according to the type information, and controlling the robotic arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information;

[0062] According to the depth map sequence, the three-dimensional motion trajectory of the components is calculated based on the adaptive optical flow algorithm, and the motion parameters of the robotic arm are dynamically optimized until the position and angle of the components to be placed meet the preset placement accuracy requirements; when the position and angle of the components to be placed meet the preset placement accuracy requirements, the robotic arm is controlled to place the components to be placed at the target position on the circuit board.

[0063] In an optional embodiment, determining a three-dimensional point cloud model and a depth map of a component to be mounted based on the multi-angle image information, performing three-dimensional feature matching on the component to be mounted according to a preset component feature library, and obtaining type information of the component to be mounted includes:

[0064] Obtaining point cloud data of the component to be identified at multiple viewing angles, and extracting local feature descriptors of the point cloud data at each viewing angle, wherein the local feature descriptors include principal curvature eigenvalues, normal vector direction angles, and local curvature radius;

[0065] Calculate the point-to-point feature similarity weight based on the local feature descriptor, substitute the feature similarity weight into the minimization registration error function, solve the rotation matrix and translation vector of the point cloud registration, and align and fuse the point cloud data under multiple perspectives;

[0066] A multi-scale feature histogram is constructed for the fused point cloud data. For each point in the point cloud data, the local geometric features of the point in different scale neighborhoods are extracted. The local geometric features are weighted statistically analyzed based on the Gaussian kernel function to obtain the global feature representation of the point.

[0067] A feature matching network based on the attention mechanism is established, the global feature representation is input into the feature matching network, the feature correspondence score is calculated through query mapping and key-value mapping, a feature matching metric is constructed in combination with regularization constraints, and the feature matching metric is matched with a preset component feature library to obtain the type information of the component to be identified.

[0068] Multi-angle acquisition equipment is used to obtain multi-view images of components to be mounted, and point cloud data of components to be identified under multiple view angles is obtained.

[0069] For the collected point cloud data, the present invention uses a local feature descriptor extraction method for feature representation. For each point cloud under each viewing angle, the principal curvature eigenvalue, normal vector direction angle, and local curvature radius are calculated for each point in the point cloud. The principal curvature eigenvalue is obtained by calculating the eigenvalue of the covariance matrix within the local neighborhood of the point cloud, representing the degree of curvature of the surface where the point is located; the normal vector direction angle is obtained by calculating the angle between the point normal vector and the reference coordinate axis, representing the orientation of the surface; and the local curvature radius is calculated by fitting the spherical curvature within the neighborhood of the point and is used to represent the local shape change rate.

[0070] In practical applications, for a resistor element, its principal curvature characteristic values ​​show obvious differences between the end and middle regions, the normal vector direction angle shows regular changes on the surface of the cylinder, and the local curvature radius can reflect the transition characteristics between the two ends and the middle of the resistor.

[0071] To achieve accurate fusion of multi-view point clouds, this paper calculates point-pair feature similarity weights based on local feature descriptors. For corresponding point pairs in point clouds from different viewpoints, the similarity scores between their local feature descriptors are calculated. In this implementation, the similarity is calculated using a weighted Euclidean distance metric, assigning a weight of 0.4 to the principal curvature eigenvalue, a weight of 0.35 to the normal vector direction angle, and a weight of 0.25 to the local curvature radius.

[0072] These feature similarity weights are then substituted into a registration error minimization function, and the rotation matrix and translation vector for point cloud registration are calculated using an iterative closest point algorithm. When processing capacitor components, this method effectively aligns the pins and the main body of the capacitor, achieving precise registration even in the presence of partial occlusion, achieving a final registration accuracy of less than 0.05mm.

[0073] After completing point cloud alignment and fusion, the present invention constructs a multi-scale feature histogram for the fused point cloud data to obtain a more comprehensive representation of component geometric features. For each point in the point cloud data, a spherical neighborhood radius of multiple scales is set: 1mm, 2mm, and 3mm. Within each scale neighborhood, local geometric features are extracted, including point density distribution, surface change rate, and principal direction consistency. A Gaussian kernel function is used to perform weighted statistics on these local geometric features, with the kernel function bandwidth parameter set to 0.4 times the neighborhood radius.

[0074] In this way, the closer the neighborhood points are to the center point, the greater their contribution to the feature, resulting in a more accurate global feature representation of the point. For an integrated circuit chip, the multi-scale feature histogram of its edge region will show obvious linear edge features, while the central region will show planar features, and the pins will have unique small-scale high curvature features.

[0075] To achieve high-precision component recognition, this paper establishes a feature matching network based on an attention mechanism. This network comprises a feature encoding layer, an attention calculation layer, and a matching metric layer. The global feature representation is input into the feature encoding layer, where a multilayer perceptron maps the feature dimensions from the original 128 dimensions to a higher-dimensional space of 256 dimensions.

[0076] In the attention calculation layer, feature correspondence scores are calculated using query mapping and key-value mapping. The query mapping converts the features of the component to be identified into a query vector, while the key-value mapping converts the component features in the feature library into key-value pairs. The dot product between the query vector and the key vector is calculated and softmax normalized to obtain the attention weight, which represents the correlation between the feature to be identified and each feature in the feature library. The weighted sum of the attention weight and the value vector constitutes the feature match representation.

[0077] In the feature matching metric calculation, a cosine similarity and structural consistency regularization constraint are combined to construct the feature matching metric. The structural consistency constraint ensures consistent matching of the overall component geometry and imposes a penalty weight for incorrect matches. In practical applications, when identifying a four-pin operational amplifier, the matching network accurately captures its unique trapezoidal package characteristics and pinout, achieving a match of over 95% with the most similar example in the feature library, significantly higher than the match with other component types.

[0078] By comparing the feature matching metric with a pre-set component signature library, the component type with the highest match is selected as the identification result. The component signature library contains signature templates for common components such as resistors, capacitors, inductors, diodes, transistors, and integrated circuits, with at least 100 sample representations for each type. In actual testing, this method achieved an accuracy rate of 98.7% for common SMD components and 96.2% for complexly packaged integrated circuits, meeting the high-precision requirements of automated placement systems.

[0079] Figure 2 This is a bar chart comparing component recognition performance based on multi-angle three-dimensional point clouds according to an embodiment of the present invention:

[0080] This image shows comparative data on four performance metrics for three different technical solutions. The figure compares the performance of traditional feature matching, point cloud registration fusion, and attention feature matching in four dimensions: recognition accuracy, processing speed, robustness, and small component recognition. The data shows that the attention feature matching solution performs best across all metrics: recognition accuracy reaches 95.6%, significantly higher than the 82.3% of the traditional solution and 88.9% of the point cloud solution; processing speed reaches 87.2%, exceeding the 65.7% of the traditional solution and 79.4% of the point cloud solution; robustness reaches 91.8%, significantly better than the 70.5% of the traditional solution and 85.3% of the point cloud solution; and small component recognition capability reaches 89.5%, also exceeding the 63.2% of the traditional solution and 72.8% of the point cloud solution. These data fully demonstrate that attention feature matching technology has significant advantages across all performance metrics.

[0081] In an optional embodiment, a region growing algorithm is used to segment the three-dimensional point cloud model of the component to be mounted, the geometric features and surface texture features of the component are extracted, and the pose estimation of the component is performed in combination with an iterative closest point algorithm, including:

[0082] Calculating the neighborhood covariance matrix eigenvalues ​​of each point in the three-dimensional point cloud data corresponding to the three-dimensional point cloud model, constructing a point cloud curvature index evaluation function based on the eigenvalues, and selecting the point with the local minimum curvature as the seed point for region growth;

[0083] A region growing criterion function is constructed based on the seed point, wherein the region growing criterion function includes a normal vector difference term, a curvature difference term, and a texture feature difference term. The three-dimensional point cloud model is segmented by calculating the normal vector angle, curvature change rate, and texture feature similarity between the point to be processed and the grown region.

[0084] The local texture descriptor is extracted from the segmented point cloud area, and the Gaussian curvature of the point cloud area is calculated as the local geometric descriptor. According to the local texture descriptor and the local geometric descriptor, point pair registration is performed through the iterative nearest point algorithm to obtain the rotation matrix and translation vector of the component to be processed, and the rotation matrix and the translation vector are used as the pose estimation result of the component to be processed.

[0085] The region growing algorithm is used to segment the 3D point cloud model of the components to be mounted, and the geometric features and surface texture features of the components are extracted. Then, the pose of the components is estimated by combining the iterative closest point algorithm.

[0086] For the acquired 3D point cloud model data, we first need to calculate the eigenvalues ​​of the neighborhood covariance matrix for each point in the point cloud. For each point in the point cloud, we select all points within a 5mm radius of that point to form a local neighborhood point set. For this neighborhood point set, we calculate its covariance matrix and perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues: λ1, λ2, and λ3 (assuming λ1 ≥ λ2 ≥ λ3).

[0087] Based on these eigenvalues, a point cloud curvature index evaluation function C = λ3 / (λ1+λ2+λ3) is constructed. This curvature index C reflects the curvature variation of the local area where the point is located. The smaller the C value, the smaller the curvature, that is, the flatter the surface. By comparing the curvature index of points in the local area, the point with the local minimum curvature is selected as the seed point for region growth. For example, in practical applications, points with a curvature index value less than 0.01 are selected as candidate seed points, and the local minimum point is selected from them as the final seed point.

[0088] A region growing criterion function is constructed based on the selected seed point. This criterion function consists of three key components: normal vector difference, curvature difference, and texture feature difference. The normal vector difference is measured by calculating the angle between the normal vectors of the processing point and the seed point (or the center point of the grown region). When the angle is less than a preset threshold (e.g., 15 degrees), the normal vector difference is considered to meet the growth conditions.

[0089] The curvature difference term is measured by calculating the curvature change rate between the processing point and the seed point. When the curvature change rate is less than a preset threshold (such as 0.05), the curvature difference is considered to meet the growth condition. The texture feature difference term is measured by calculating the texture feature similarity between the processing point and the seed point. Here, the local binary pattern is used as the texture feature descriptor. When the Hamming distance of the feature descriptor is less than a preset threshold (such as 20), the texture difference is considered to meet the growth condition.

[0090] The region growing process starts from the seed point and gradually adds the adjacent points that meet the growth criteria to the current region. For each point in the point cloud, check whether it meets the growth criteria with the current growing region. Specifically, calculate the angle between the normal vector of the point and the nearest point in the current region. If the angle is less than 15 degrees; calculate the curvature change rate between the point and the nearest point. If the change rate is less than 0.05; calculate the texture feature Hamming distance between the point and the nearest point. If the distance is less than 20, then add the point to the current growing region. This process continues until no new points can be added to the current region. Then, reselect the seed point from the remaining unsegmented points and start a new region growing process until all points are assigned to the corresponding regions.

[0091] For the segmented point cloud region, local texture and geometry descriptors are extracted. The local texture descriptor uses a local binary pattern. After the point cloud is projected onto a two-dimensional image, the pixel values ​​within the neighborhood of each point are binarized to form a binary code describing the local texture features of that point. The local geometry descriptor uses Gaussian curvature. By calculating the principal curvatures k1 and k2 of the local surface of the point cloud, the Gaussian curvature K = k1 × k2 is obtained. Gaussian curvature effectively describes the local geometric shape characteristics of a surface. For planar regions, K ≈ 0, for convex regions, K > 0, and for concave regions, K < 0.

[0092] To estimate the pose of the component to be placed, an iterative closest point algorithm is used for point cloud registration. Based on the extracted local texture and geometric descriptors, this algorithm establishes a correspondence between the component point cloud to be processed and the template point cloud. In a real-world example, for a rectangular component, 5,000 feature points were first extracted from the segmented point cloud. Each feature point includes its position coordinates, normal vector, Gaussian curvature value, and local binary pattern descriptor.

[0093] By comparing the characteristic points of the component to be processed with those of the template component, 150 initial corresponding point pairs were established. Based on these corresponding point pairs, the iterative closest point algorithm optimized the transformation matrix by minimizing the sum of the squared distances between the point pairs. After 25 iterations, the algorithm converged to obtain the rotation matrix R and translation vector T, where R is expressed as Euler angles (α=2.3°, β=-1.5°, γ=0.8°) and T=(5.2mm, -3.7mm, 0.5mm). This result indicates the rotation and translation of the component to be placed relative to the template position and can be used directly to guide the precise placement of the component.

[0094] Through the above method, the precise segmentation and pose estimation of the three-dimensional point cloud of the components to be mounted are achieved, providing accurate component position and pose information for the automated placement equipment, thereby improving the placement accuracy and efficiency.

[0095] In an optional embodiment, calling preset component placement parameters according to the type information, and controlling the robot arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information includes:

[0096] Extracting corresponding placement reference parameters from a component standard parameter library according to the type information, wherein the placement reference parameters include the spatial coordinates of the target placement position, the range of the allowable placement angle, the limit value of the maximum placement pressure, and the judgment standard of the placement accuracy;

[0097] Determine the current spatial position and attitude angle of the component to be mounted, calculate the coordinate difference between the current spatial position and the target mounting position, compare the attitude angle with the range of the allowable mounting angle, and generate a position compensation vector and an angle correction value;

[0098] planning the motion trajectory of the manipulator according to the position compensation vector and the angle correction value, decomposing the motion trajectory into a horizontal displacement adjustment phase and a vertical posture adjustment phase, and calculating motion parameters of the manipulator joints in the two phases respectively;

[0099] The position error and angle error of the component to be mounted relative to the target mounting position are continuously detected. When the position error and the angle error both meet the judgment criteria of the mounting accuracy, the current motion state of the robot arm is locked and the final mounting action is performed.

[0100] like Figure 3 As shown, the method includes:

[0101] Obtain the type and posture deviation information of the components to be mounted. Type information, such as resistors, capacitors, integrated circuits, etc., can be obtained through a visual recognition system or barcode scanning system. Posture deviation information is captured by a high-precision camera with the current position and angle of the component. The difference from the standard position is calculated using an image processing algorithm.

[0102] Based on the acquired type information, the corresponding placement benchmark parameters are extracted from a pre-established component standard parameter library. This parameter library stores the standard placement parameters for each type of component, including the spatial coordinates of the target placement position (X, Y, and Z coordinates), the allowable placement angle range (such as ±2 degrees), the maximum placement pressure limit (such as 0.5N for capacitors and 0.8N for integrated circuits), and the placement accuracy criteria (such as position accuracy ±0.05mm and angle accuracy ±0.5 degrees).

[0103] For example, for a 0603 chip resistor, the target mounting position spatial coordinates are (120.45mm, 85.67mm, 0.00mm), the allowable mounting angle range is -1.5 degrees to +1.5 degrees, the maximum mounting pressure is 0.3N, the mounting position accuracy requirement is ±0.03mm, and the angle accuracy requirement is ±0.3 degrees.

[0104] After obtaining the placement reference parameters, the system determines the current spatial position and posture angle of the component to be placed. A high-precision vision system is used to obtain the actual position of the component on the nozzle. For example, if the current spatial position is (120.52mm, 85.59mm, 0.05mm) and the current posture angle is 1.2 degrees, the system calculates the coordinate difference between the current spatial position and the target placement position, deriving a position compensation vector (-0.07mm, 0.08mm, -0.05mm). The system also compares the current posture angle of 1.2 degrees with the allowable placement angle range of -1.5 degrees to +1.5 degrees. To confirm that the angle is within the allowable range, an angle correction of -1.2 degrees is generated (correcting the angle to 0 degrees, the standard placement angle).

[0105] Based on the calculated position compensation vector and angle correction, the system plans the motion trajectory of the robot arm. The motion trajectory is decomposed into a horizontal displacement adjustment phase and a vertical posture adjustment phase. In the horizontal displacement adjustment phase, the robot arm moves in the XY plane to adjust the horizontal position of the component to just above the target position. For the above case, the robot arm moves -0.07mm in the X-axis direction and 0.08mm in the Y-axis direction. In the vertical posture adjustment phase, the robot arm adjusts the Z-axis height and rotates the nozzle angle. In this case, the Z-axis is lowered by 0.05mm, and the nozzle is rotated -1.2 degrees to achieve angle correction.

[0106] The robot's motion parameter calculations take acceleration and speed control into account. During the horizontal displacement adjustment phase, the system sets a maximum movement speed of 50 mm / s and an acceleration of 200 mm / s², calculating the actual motion parameters based on the magnitude of the position compensation vector. During the vertical posture adjustment phase, the Z-axis descent speed is set to 10 mm / s and the angular rotation speed is set to 5 degrees / s to ensure a smooth transition to placement.

[0107] The positional and angular errors of components to be placed relative to the target placement position are continuously monitored. The high-precision vision system captures images at a 50Hz frequency, calculating in real time the Euclidean distance between the component's current position (X', Y', Z') and the target position (X, Y, Z). It also calculates the difference between the current angle θ' and the target angle θ. When the positional error is less than the placement accuracy requirement (±0.03mm) and the angular error is less than the angular accuracy requirement (±0.3 degrees), the system determines that the adjustment has met the accuracy requirements. In the above case, after precise adjustment, the final positional error was 0.02mm and the angular error was 0.2 degrees, meeting the placement accuracy criteria.

[0108] The robot arm's current motion state is locked, fine-tuning its position and angle is stopped, and the final placement action is executed. This action involves controlling the robot arm to descend vertically to the placement surface at a constant speed of 2mm / s, applying a placement pressure of 0.25N (less than the maximum allowable pressure of 0.3N), maintaining the pressure for 0.2 seconds to ensure a secure placement, then closing the nozzle vacuum and vertically raising the robot arm 5mm to complete the placement process.

[0109] Through the above steps, the system can call the corresponding placement parameters according to the characteristics of different types of components, and combine the real-time detected posture deviation information to accurately control the robotic arm to adjust the position and angle of components, thereby achieving high-precision and high-reliability component placement, effectively improving the quality and efficiency of electronic product manufacturing.

[0110] In an optional embodiment, according to the depth map sequence, the three-dimensional motion trajectory of the component is calculated based on an adaptive optical flow algorithm, and the motion parameters of the robot arm are dynamically optimized until the position and angle of the component to be placed meet the preset placement accuracy requirements, including:

[0111] Performing multi-scale decomposition on the depth map sequence to obtain depth feature maps at different resolutions, and calculating spatial position information of components to be mounted based on the depth feature maps;

[0112] Based on the grayscale change relationship between the depth feature maps of adjacent frames, an optical flow displacement calculation rule is established. The optical flow displacement calculation rule includes grayscale consistency constraints and motion smoothness constraints. Adaptive adjustment of local motion estimation is achieved by adjusting the constraint coefficient;

[0113] Combining the spatial position information and the optical flow displacement calculation results, analyzing the depth value differences of corresponding points in the depth feature maps of adjacent frames, determining the depth change weights, and correcting the displacement calculation results according to the depth change weights to obtain the three-dimensional motion parameters of the components to be mounted;

[0114] Constructing a motion trajectory of the component according to the three-dimensional motion parameters, analyzing the acceleration changes at adjacent moments in the motion trajectory, and reducing the acceleration fluctuation of the trajectory curve through iterative optimization to obtain smooth motion trajectory data;

[0115] The smooth motion trajectory data is converted into the robot arm joint motion parameters, the motion state of the robot arm end effector is calculated based on the joint motion parameters, and the position and posture information of the components to be mounted are collected in real time. When the position and posture information meet the preset mounting accuracy requirements, the component mounting operation is executed.

[0116] According to the depth map sequence, the three-dimensional motion trajectory of the components can be calculated based on the adaptive optical flow algorithm, and the motion parameters of the robotic arm can be dynamically optimized.

[0117] Depth map sequences are decomposed at multiple scales, using a Gaussian pyramid algorithm for image downsampling, typically with four to five scale levels. For example, for an original depth map with a resolution of 640×480, downsampled images of 320×240, 160×120, 80×60, and 40×30 are generated. The Sobel operator is applied to the depth map at each scale level for edge extraction, enhancing the saliency of depth features.

[0118] After feature extraction, the two-dimensional pixel coordinates and depth values ​​are converted into three-dimensional spatial coordinates through projection transformation. The camera's intrinsic parameter matrix can be obtained through calibration. For example, if the focal length is 525 pixels, the principal point coordinates are (320, 240). For each pixel point (u, v) in the image, its depth value is d, and the three-dimensional position (X, Y, Z) in the world coordinate system can be calculated, thereby obtaining the spatial position information of the component to be mounted.

[0119] When establishing the optical flow displacement calculation rules between depth feature maps of adjacent frames, the objective function is constructed based on grayscale consistency constraints and motion smoothness constraints. Grayscale consistency constraints ensure that corresponding points have similar grayscale values ​​in different time frames, which manifests as continuity of depth values. Motion smoothness constraints ensure that the changes in motion vectors within a local area are smooth.

[0120] For component edge regions, the grayscale consistency constraint weight can be set to 0.6, and the motion smoothness constraint weight to 0.4. Within component interiors, the grayscale consistency constraint weight is adjusted to 0.8, and the motion smoothness constraint weight is adjusted to 0.2. In implementation, an iterative optimization method is used to calculate pixel displacement within a 5×5 local window. The number of iterations is set to 20, and the algorithm is terminated early when the displacement change between adjacent iterations is less than 0.01 pixels, improving computational efficiency.

[0121] When correcting motion parameters by combining spatial position information with the results of optical flow displacement calculations, it is necessary to analyze the depth differences between corresponding points in the depth feature maps of adjacent frames. For each matching point pair, the absolute value of the depth difference is calculated. If the difference is greater than a preset threshold (e.g., 10 mm), it indicates occlusion or mismatch, and the weight of that point in the displacement calculation should be reduced.

[0122] In practice, an exponential decay function can be used to set the depth change weight. When the depth difference is 5 mm, the weight is 0.9, when the depth difference is 15 mm, the weight drops to 0.3, and when the depth difference exceeds 30 mm, the weight approaches 0. These weights are applied to the displacement vectors for weighted averaging, eliminating the influence of outliers and obtaining more accurate 3D motion parameters, including translation vectors (Tx, Ty, Tz) and rotation angles (Rx, Ry, Rz).

[0123] When constructing component motion trajectories based on 3D motion parameters, we use cubic spline interpolation to generate a continuous, smooth trajectory curve. Ten intermediate points are interpolated between every two key points to create evenly sampled trajectory data. To optimize acceleration fluctuations, we calculate the acceleration value of each point along the trajectory, setting an acceleration threshold of 0.2 m / s². When acceleration exceeds the threshold, we perform local smoothing by adjusting the positions of adjacent control points.

[0124] The optimization process is iterative, with the maximum distance the control point moves in each iteration limited to 5% of its original position. Iterations terminate when the maximum acceleration falls below a threshold or when 50 iterations have been reached. The optimized trajectory curve exhibits lower acceleration fluctuations, effectively reducing jitter during robotic arm movement.

[0125] To convert smoothed motion trajectory data into robot joint motion parameters, an inverse kinematics algorithm is used to calculate the joint angles. Assuming a six-axis robot, the robot's kinematic model is represented by DH parameters, with joint limits set to ±180 degrees. A minimum jerk optimization algorithm is applied to the calculated joint angle sequence to generate the joint motion trajectory, with a sampling interval of 10 milliseconds.

[0126] The robotic arm control system receives joint angle commands in real time and achieves precise tracking through a PID controller, with the proportional gain Kp set to 120, the integral gain Ki to 0.5, and the differential gain Kd to 15. Simultaneously, a visual sensor mounted at the end of the robotic arm collects the position and posture information of the components to be placed in real time. When the position deviation is less than 0.05 mm and the angle deviation is less than 0.1 degrees, it is determined that the preset placement accuracy requirements are met, and the component placement operation is then executed, completing precise positioning and installation.

[0127] In an optional embodiment, combining the spatial position information and the optical flow displacement calculation results, analyzing the depth value differences of corresponding points in the depth feature maps of adjacent frames, determining the depth change weight, and correcting the displacement calculation results according to the depth change weight to obtain the three-dimensional motion parameters of the components to be mounted, including:

[0128] The depth feature map in the previous time series is set as the reference frame, and the depth feature map in the next time series is set as the current frame;

[0129] Detecting feature points in the reference frame, performing descriptor extraction on the feature points, searching for corresponding feature points in the current frame based on the descriptors, establishing feature point correspondences, and calculating spatial position changes of the feature points based on the feature point correspondences;

[0130] An optical flow tracking algorithm is used to calculate the position offset of the feature point in the reference frame in the current frame, and the position offset is converted into a three-dimensional spatial displacement in combination with camera parameters to obtain optical flow motion information of the feature point;

[0131] Extracting depth values ​​of corresponding feature points in the reference frame and the current frame, calculating a depth difference between each pair of feature points, and setting a depth weight based on the depth difference, wherein the depth weight is used to characterize the reliability of the depth measurement of the feature points;

[0132] Performing depth-weighted fusion on the spatial position change and the optical flow motion information, wherein the fusion weight is determined by the depth weight, and obtaining a motion vector of the feature point through weighted fusion;

[0133] The rigid body motion parameters of the component to be mounted are estimated based on the motion vector, the rigid body motion parameters are decomposed into rotation components and translation components, and the three-dimensional motion parameters of the component to be mounted are output.

[0134] To address the visual guidance issues during component placement on chip placement machines, a 3D motion parameter estimation method based on depth information and optical flow analysis is proposed. This method combines the spatial position information acquired by a depth camera with the displacement results calculated by an optical flow algorithm. By analyzing the depth differences between corresponding points in the depth feature maps of adjacent frames, this method determines the depth change weights and corrects the displacement calculations to accurately obtain the 3D motion parameters of the components being placed.

[0135] Acquire depth feature maps from adjacent time series. The first depth feature map acquired in time is defined as the reference frame, and the second one acquired later is defined as the current frame. For example, at a 30Hz acquisition frequency, the interval between adjacent frames is 33.3 milliseconds. Both depth feature maps contain images of the component to be mounted and the corresponding depth information.

[0136] Feature points were detected using the Harris corner detection algorithm. A corner response threshold of 0.01 and a non-maximum suppression window size of 3 pixels were set, resulting in 500 feature points. These feature points were characterized using the ORB descriptor, each containing a 256-bit binary code. A search window (21×21 pixels in size around the reference frame feature point) was then set in the current frame, centered around the reference frame feature point. The Hamming distance matching algorithm was then used to find the corresponding feature point within this window.

[0137] When the Hamming distance is less than a threshold of 30, a matching point is considered found, and a feature point correspondence is established. Based on this established feature point correspondence, the position difference of each pair of feature points in the image coordinate system is calculated. Combined with the camera's intrinsic parameters (e.g., focal length of 525 pixels and principal point coordinates of (320, 240)), this two-dimensional position difference is converted into a three-dimensional spatial position change.

[0138] The Lucas-Kanade optical flow algorithm is used to calculate the precise position of feature points in the reference frame in the current frame. Specifically, a 15×15 pixel image block is captured, centered around each feature point in the reference frame. The optimal matching position is searched for near the corresponding position in the current frame. The iterative solution is set to a maximum of 20 iterations and a termination threshold of 0.001 pixels. The displacement vector of the feature point in the image plane is calculated through optical flow. Combined with the calibration parameters of the depth camera (such as a camera baseline length of 75 mm and a parallax accuracy of 0.1 pixel), the two-dimensional displacement is converted into a three-dimensional spatial displacement to obtain the optical flow motion information of the feature point.

[0139] Extract the depth values ​​of corresponding feature point locations from the reference frame and the current frame. For example, if the depth value of a feature point in the reference frame is 500 mm and the depth value of the corresponding point in the current frame is 510 mm, the calculated depth difference is 10 mm. For each pair of feature points, a depth weight is assigned based on the depth difference. When the depth difference is less than 5 mm, the depth weight is set to 1.0; when the depth difference is between 5 and 15 mm, the depth weight decreases linearly from 1.0 to 0.5; when the depth difference is greater than 15 mm, the depth weight is set to 0.5. This gives points with stable depth measurements a higher weight, while points with large depth fluctuations receive a lower weight.

[0140] After obtaining the spatial position change and optical flow motion information, depth-weighted fusion is performed. The fusion formula is: the final motion vector of the feature point is equal to the depth weight multiplied by the spatial position change, plus (1-depth weight) multiplied by the optical flow motion information. For example, if a feature point has a depth weight of 0.8, a spatial position change of (2, 3, 1) mm, and optical flow motion information of (2.2, 3.3, 1.1) mm, the fused motion vector is (2.04, 3.06, 1.02) mm. This weighted fusion strategy leverages the strengths of both measurement methods to improve motion estimation accuracy.

[0141] After calculating the feature point motion vectors, the RANSAC algorithm is used to estimate the rigid body motion parameters of the component to be placed. The maximum number of RANSAC iterations is set to 1000, and the inlier threshold is set to 0.5 mm. In each iteration, three pairs of feature point correspondences are randomly selected, the optimal rigid body transformation matrix is ​​calculated, and the number of inliers that meet this transformation is counted. After multiple iterations, the transformation matrix with the most inliers is selected as the final result. This rigid body transformation matrix is ​​decomposed into a rotation matrix and a translation vector. The rotation matrix can be further converted into Euler angles (e.g., 0.5 degree rotation around the X-axis, -0.2 degree rotation around the Y-axis, and 0.1 degree rotation around the Z-axis). The translation vector directly represents the translation in three directions (e.g., 2.5 mm in the X-axis, -1.2 mm in the Y-axis, and 0.8 mm in the Z-axis). These rotation and translation parameters together constitute the three-dimensional motion parameters of the component to be placed and can be directly used in the placement machine control system for precise component positioning and placement.

[0142] Through the above method, in actual tests, in the component tracking experiment with a movement speed of 10 mm / s, the average position estimation error is less than 0.2 mm, and the average angle estimation error is less than 0.1 degrees, meeting the high-precision placement requirements.

[0143] Figure 4 This is a bar chart comparing the performance of the component 3D motion parameter estimation method based on deep feature fusion according to an embodiment of the present invention:

[0144] This image shows the comparative data analysis of three different technical methods (feature point matching method, optical flow tracking method and depth weighted fusion method) on five performance indicators. The data shows that in terms of translation accuracy, the deep weighted fusion method achieved the highest level of 92.8%, significantly outperforming the 83.7% of the optical flow tracking method and the 75.3% of the feature point matching method. In terms of rotation accuracy, the deep weighted fusion method also performed best, reaching 89.6%, higher than the 81.4% of the feature point matching method and the 77.5% of the optical flow tracking method. In terms of motion estimation stability, the deep weighted fusion method achieved 93.1%, far exceeding the 80.3% of the optical flow tracking method and the 68.7% of the feature point matching method. In terms of noise robustness, the deep weighted fusion method achieved 87.4%, significantly higher than the 73.6% of the optical flow tracking method and the 66.2% of the feature point matching method. In terms of edge component tracking, the deep weighted fusion method still led with 85.3%, showing a clear advantage over the 67.2% of the optical flow tracking method and the 58.9% of the feature point matching method. Overall, the deep weighted fusion method demonstrated the best performance across all evaluation metrics.

[0145] A second aspect of an embodiment of the present invention provides a system for accurately positioning and placing components on a circuit board based on vision guidance, comprising:

[0146] The first unit is configured to obtain multi-angle image information of components to be mounted, determine a three-dimensional point cloud model and a depth map sequence of the components to be mounted based on the multi-angle image information, perform three-dimensional feature matching on the components to be mounted according to a preset component feature library, and obtain type information of the components to be mounted;

[0147] The second unit is used to segment the three-dimensional point cloud model of the components to be mounted using the region growing algorithm, extract the geometric features and surface texture features of the components, and estimate the pose of the components in combination with the iterative closest point algorithm to obtain the pose deviation information of the components to be mounted relative to the preset mounting reference point;

[0148] A third unit is configured to call preset component placement parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information;

[0149] The fourth unit is used to calculate the three-dimensional motion trajectory of the components based on the adaptive optical flow algorithm according to the depth map sequence, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the components to be placed meet the preset placement accuracy requirements; when the position and angle of the components to be placed meet the preset placement accuracy requirements, the robotic arm is controlled to place the components to be placed at the target position on the circuit board.

[0150] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0151] processor;

[0152] a memory for storing processor-executable instructions;

[0153] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0154] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0155] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for accurately positioning and mounting components on a circuit board based on vision guidance, characterized in that: include: Acquire multi-angle image information of components to be mounted, determine a three-dimensional point cloud model and a depth map sequence of the components to be mounted based on the multi-angle image information, perform three-dimensional feature matching on the components to be mounted according to a preset component feature library, and obtain type information of the components to be mounted, including: Obtain point cloud data of the component to be identified at multiple viewing angles, extract local feature descriptors of the point cloud data at each viewing angle, the local feature descriptors including principal curvature eigenvalues, normal vector direction angles, and local curvature radius; calculate point-pair feature similarity weights based on the local feature descriptors, substitute the feature similarity weights into a registration error minimization function, solve the rotation matrix and translation vector for point cloud registration, and align and fuse the point cloud data at multiple viewing angles; A multi-scale feature histogram is constructed for the fused point cloud data. For each point in the point cloud data, the local geometric features of the point in different scale neighborhoods are extracted. The local geometric features are weighted statistically analyzed based on the Gaussian kernel function to obtain the global feature representation of the point. A feature matching network based on the attention mechanism is established, and the global feature representation is input into the feature matching network. The feature correspondence score is calculated through query mapping and key-value mapping. A feature matching metric is constructed in combination with regularization constraints. The feature matching metric is matched with a preset component feature library to obtain the type information of the component to be identified. The region growing algorithm is used to segment the 3D point cloud model of the components to be mounted, extract the geometric features and surface texture features of the components, and combine it with the iterative closest point algorithm to estimate the pose of the components to be mounted, and obtain the pose deviation information of the components to be mounted relative to the preset mounting reference point; Calling preset component placement parameters according to the type information, and controlling the robotic arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information; According to the depth map sequence, the three-dimensional motion trajectory of the components is calculated based on the adaptive optical flow algorithm, and the motion parameters of the robotic arm are dynamically optimized until the position and angle of the components to be placed meet the preset placement accuracy requirements; when the position and angle of the components to be placed meet the preset placement accuracy requirements, the robotic arm is controlled to place the components to be placed at the target position on the circuit board.

2. The method according to claim 1, characterized in that The region growing algorithm is used to segment the 3D point cloud model of the components to be mounted, extract the geometric features and surface texture features of the components, and combine iterative closest point algorithm to estimate the pose of the components. The following steps are involved: Calculating the neighborhood covariance matrix eigenvalues ​​of each point in the three-dimensional point cloud data corresponding to the three-dimensional point cloud model, constructing a point cloud curvature index evaluation function based on the eigenvalues, and selecting the point with the local minimum curvature as the seed point for region growth; A region growing criterion function is constructed based on the seed point, wherein the region growing criterion function includes a normal vector difference term, a curvature difference term, and a texture feature difference term. The three-dimensional point cloud model is segmented by calculating the normal vector angle, curvature change rate, and texture feature similarity between the point to be processed and the grown region. The local texture descriptor is extracted from the segmented point cloud area, and the Gaussian curvature of the point cloud area is calculated as the local geometric descriptor. According to the local texture descriptor and the local geometric descriptor, point pair registration is performed through the iterative nearest point algorithm to obtain the rotation matrix and translation vector of the component to be processed, and the rotation matrix and the translation vector are used as the pose estimation result of the component to be processed.

3. The method according to claim 1, characterized in that Calling preset component placement parameters according to the type information, and controlling the robot arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information includes: Extracting corresponding placement reference parameters from a component standard parameter library according to the type information, wherein the placement reference parameters include the spatial coordinates of the target placement position, the range of the allowable placement angle, the limit value of the maximum placement pressure, and the judgment standard of the placement accuracy; Determine the current spatial position and attitude angle of the component to be mounted, calculate the coordinate difference between the current spatial position and the target mounting position, compare the attitude angle with the range of the allowable mounting angle, and generate a position compensation vector and an angle correction value; planning the motion trajectory of the manipulator according to the position compensation vector and the angle correction value, decomposing the motion trajectory into a horizontal displacement adjustment phase and a vertical posture adjustment phase, and calculating motion parameters of the manipulator joints in the two phases respectively; The position error and angle error of the component to be mounted relative to the target mounting position are continuously detected. When the position error and the angle error both meet the judgment criteria of the mounting accuracy, the current motion state of the robot arm is locked and the final mounting action is performed.

4. The method according to claim 1, wherein According to the depth map sequence, the three-dimensional motion trajectory of the component is calculated based on the adaptive optical flow algorithm, and the motion parameters of the robot arm are dynamically optimized until the position and angle of the component to be placed meet the preset placement accuracy requirements, including: Performing multi-scale decomposition on the depth map sequence to obtain depth feature maps at different resolutions, and calculating spatial position information of components to be mounted based on the depth feature maps; Based on the grayscale change relationship between the depth feature maps of adjacent frames, an optical flow displacement calculation rule is established. The optical flow displacement calculation rule includes grayscale consistency constraints and motion smoothness constraints. Adaptive adjustment of local motion estimation is achieved by adjusting the constraint coefficient; Combining the spatial position information and the optical flow displacement calculation results, analyzing the depth value differences of corresponding points in the depth feature maps of adjacent frames, determining the depth change weights, and correcting the displacement calculation results according to the depth change weights to obtain the three-dimensional motion parameters of the components to be mounted; Constructing a motion trajectory of the component according to the three-dimensional motion parameters, analyzing the acceleration changes at adjacent moments in the motion trajectory, and reducing the acceleration fluctuation of the trajectory curve through iterative optimization to obtain smooth motion trajectory data; The smooth motion trajectory data is converted into the robot arm joint motion parameters, the motion state of the robot arm end effector is calculated based on the joint motion parameters, and the position and posture information of the components to be mounted are collected in real time. When the position and posture information meet the preset mounting accuracy requirements, the component mounting operation is executed.

5. The method according to claim 4, characterized in that Combining the spatial position information and the optical flow displacement calculation results, analyzing the depth value differences of corresponding points in the depth feature maps of adjacent frames, determining the depth change weight, and correcting the displacement calculation results according to the depth change weight to obtain the three-dimensional motion parameters of the components to be mounted, including: The depth feature map in the previous time series is set as the reference frame, and the depth feature map in the next time series is set as the current frame; Detecting feature points in the reference frame, performing descriptor extraction on the feature points, searching for corresponding feature points in the current frame based on the descriptors, establishing feature point correspondences, and calculating spatial position changes of the feature points based on the feature point correspondences; An optical flow tracking algorithm is used to calculate the position offset of the feature point in the reference frame in the current frame, and the position offset is converted into a three-dimensional spatial displacement in combination with camera parameters to obtain optical flow motion information of the feature point; Extracting depth values ​​of corresponding feature points in the reference frame and the current frame, calculating a depth difference between each pair of feature points, and setting a depth weight based on the depth difference, wherein the depth weight is used to characterize the reliability of the depth measurement of the feature points; Performing depth-weighted fusion on the spatial position change and the optical flow motion information, wherein the fusion weight is determined by the depth weight, and obtaining a motion vector of the feature point through weighted fusion; The rigid body motion parameters of the component to be mounted are estimated based on the motion vector, the rigid body motion parameters are decomposed into rotation components and translation components, and the three-dimensional motion parameters of the component to be mounted are output.

6. A circuit board component precision positioning and placement system based on vision guidance, used to implement the method according to any one of claims 1 to 5, characterized in that: include: The first unit is configured to obtain multi-angle image information of components to be mounted, determine a three-dimensional point cloud model and a depth map sequence of the components to be mounted based on the multi-angle image information, perform three-dimensional feature matching on the components to be mounted according to a preset component feature library, and obtain type information of the components to be mounted; The second unit is used to segment the three-dimensional point cloud model of the components to be mounted using the region growing algorithm, extract the geometric features and surface texture features of the components, and estimate the pose of the components in combination with the iterative closest point algorithm to obtain the pose deviation information of the components to be mounted relative to the preset mounting reference point; A third unit is configured to call preset component placement parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be placed according to the placement parameters and the posture deviation information; The fourth unit is used to calculate the three-dimensional motion trajectory of the components based on the adaptive optical flow algorithm according to the depth map sequence, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the components to be placed meet the preset placement accuracy requirements; when the position and angle of the components to be placed meet the preset placement accuracy requirements, the robotic arm is controlled to place the components to be placed at the target position on the circuit board.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images