Circuit board component accurate positioning and mounting method and system based on visual guidance

Abstract

The invention provides a circuit board component accurate positioning and mounting method and system based on visual guidance, and relates to the technical field of mounting, and the method comprises the steps: obtaining the multi-angle image information of a to-be-mounted component, determining a three-dimensional point cloud model and a depth map sequence, and determining the type information; segmenting the point cloud model through a region growing algorithm to extract features for pose estimation; the mounting parameters are called and combined with the posture deviation information to control a mechanical arm; and motion parameters are optimized based on a self-adaptive optical flow algorithm until the precision requirement is met. According to the invention, the component mounting precision and efficiency are improved, and the mounting failure rate is reduced.

Description

Technical Field

[0001] The present invention relates to the field of mounting technology, and particularly to a method and system for accurate positioning and mounting of circuit board components based on vision guidance. Background Art

[0002] With the development of electronic products towards miniaturization and high integration, the requirements for the mounting accuracy of circuit board components are getting higher and higher. Traditional circuit board component mounting technologies mainly rely on manual operation or simple mechanical positioning systems, which cannot meet the requirements of modern electronic manufacturing for high-precision and high-efficiency mounting. Currently, circuit board component mounting mainly uses automatic pick-and-place machines. The components are identified and positioned through a vision system, and then the components are accurately mounted to the specified positions on the circuit board by a robotic arm.

[0003] However, the existing circuit board component mounting technologies still have some defects and deficiencies. First, the traditional two-dimensional vision recognition system has a low recognition accuracy when dealing with components with complex shapes or special materials. Especially for components with similar appearances but different functions, it is easy to have misrecognition, resulting in mounting errors. Second, the existing technologies mainly rely on a single algorithm for component pose estimation, and have insufficient adaptability to the pose changes of components during the picking process, making it difficult to adjust the mounting strategy in real time, thus affecting the final mounting accuracy. In addition, the traditional mounting system lacks the ability of dynamic optimization. During the process from component picking to placement, the position deviations caused by factors such as mechanical vibration and thermal expansion cannot be effectively compensated, making it difficult to meet the requirements of micron-level mounting accuracy.

[0004] With the continuous development of electronic product manufacturing processes, the requirements for the mounting accuracy and efficiency of circuit board components are also getting higher and higher. There is an urgent need for a mounting method that can achieve accurate component recognition, real-time pose estimation, and dynamic trajectory optimization to improve the mounting quality and production efficiency. Summary of the Invention

[0005] Embodiments of the present invention provide a method and system for accurate positioning and mounting of circuit board components based on vision guidance, which can solve the problems in the prior art.

[0006] In the first aspect of the embodiments of the present invention, a method for accurate positioning and mounting of circuit board components based on vision guidance is provided, including: Obtaining multi-angle image information of the component to be mounted, determining a three-dimensional point cloud model and a depth map sequence of the component to be mounted based on the multi-angle image information, and performing three-dimensional feature matching on the component to be mounted according to a preset component feature library to obtain the type information of the component to be mounted; The region growing algorithm is used to segment the three-dimensional point cloud model of the component to be mounted, extract the geometric features and surface texture features of the component, and combine the iterative closest point algorithm to estimate the pose of the component, so as to obtain the attitude deviation information of the component to be mounted relative to the preset mounting reference point; Call the preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the attitude deviation information; According to the depth map sequence, calculate the three-dimensional motion trajectory of the component based on the adaptive optical flow algorithm, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements; when the position and angle of the component to be mounted meet the preset mounting accuracy requirements, control the robotic arm to mount the component to be mounted to the target position on the circuit board.

[0007] Based on the multi-angle image information, determine the three-dimensional point cloud model and depth map of the component to be mounted, and perform three-dimensional feature matching on the component to be mounted according to the preset component feature library, and obtain the type information of the component to be mounted, including: Obtain the point cloud data of the component to be recognized from multiple perspectives, and extract the local feature descriptors of the point cloud data from each perspective. The local feature descriptors include the principal curvature eigenvalue, the normal vector direction angle, and the local curvature radius; Calculate the point pair feature similarity weight based on the local feature descriptors, substitute the feature similarity weight into the minimum registration error function, solve the rotation matrix and translation vector of point cloud registration, and align and fuse the point cloud data from multiple perspectives; Construct a multi-scale feature histogram for the fused point cloud data. For each point in the point cloud data, extract the local geometric features of the point in different scale neighborhoods, and perform weighted statistics on the local geometric features based on the Gaussian kernel function to obtain the global feature representation of the point; Establish a feature matching network based on the attention mechanism, input the global feature representation into the feature matching network, calculate the feature correspondence score through query mapping and key-value mapping, construct a feature matching metric in combination with regularization constraints, and match the feature matching metric with the preset component feature library to obtain the type information of the component to be recognized.

[0008] Using the region growing algorithm to segment the three-dimensional point cloud model of the component to be mounted, and extracting the geometric features and surface texture features of the component. Combining the iterative closest point algorithm to estimate the pose of the component includes: Calculate the neighborhood covariance matrix eigenvalue of each point in the three-dimensional point cloud data corresponding to the three-dimensional point cloud model, construct a point cloud curvature index evaluation function based on the eigenvalue, and select the point with the local minimum curvature as the seed point for region growing; Construct a region growing criterion function based on the seed points. The region growing criterion function includes a normal vector difference term, a curvature difference term, and a texture feature difference term. By calculating the normal vector angle, curvature change rate, and texture feature similarity between the points to be processed and the grown region, the 3D point cloud model is segmented into regions; Extract local texture descriptors for the segmented point cloud regions, and at the same time calculate the Gaussian curvature of the point cloud regions as local geometric descriptors; according to the local texture descriptors and the local geometric descriptors, perform point pair registration through the iterative closest point algorithm to obtain the rotation matrix and translation vector of the component to be processed, and use the rotation matrix and the translation vector as the pose estimation result of the component to be processed.

[0009] Call the preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the attitude deviation information, including: Extract the corresponding mounting reference parameters from the component standard parameter library according to the type information. The mounting reference parameters include the spatial coordinates of the target mounting position, the range interval of the allowable mounting angle, the limit value of the maximum mounting pressure, and the judgment criteria for mounting accuracy; Determine the current spatial position and attitude angle of the component to be mounted, calculate the coordinate difference between the current spatial position and the target mounting position, compare the attitude angle with the range interval of the allowable mounting angle, and generate a position compensation vector and an angle correction amount; Plan the motion trajectory of the robotic arm according to the position compensation vector and the angle correction amount, decompose the motion trajectory into a horizontal displacement adjustment stage and a vertical attitude adjustment stage, and calculate the motion parameters of the robotic arm joints in the two stages respectively; Continuously detect the position error and angle error of the component to be mounted relative to the target mounting position. When the position error and the angle error simultaneously meet the judgment criteria for mounting accuracy, lock the current motion state of the robotic arm and perform the final mounting action.

[0010] According to the depth map sequence, calculate the 3D motion trajectory of the component based on the adaptive optical flow algorithm, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements, including: Perform multi-scale decomposition on the depth map sequence to obtain depth feature maps at different resolutions, and calculate the spatial position information of the component to be mounted according to the depth feature maps; Based on the gray-scale change relationship between adjacent frame depth feature maps, establish an optical flow displacement calculation rule. The optical flow displacement calculation rule includes gray-scale consistency constraints and motion smoothness constraints, and adaptively adjusts the local motion estimation by adjusting the constraint coefficients; Combined with the spatial position information and the optical flow displacement calculation result, analyze the depth value difference of corresponding points in adjacent frame depth feature maps, determine the depth change weight, and correct the displacement calculation result according to the depth change weight to obtain the three-dimensional motion parameters of the component to be mounted; Construct the motion trajectory of the component according to the three-dimensional motion parameters, analyze the acceleration change at adjacent moments in the motion trajectory, and reduce the acceleration fluctuation of the trajectory curve through iterative optimization to obtain smooth motion trajectory data; Convert the smooth motion trajectory data into robotic arm joint motion parameters, calculate the motion state of the end effector of the robotic arm according to the joint motion parameters, and collect the position and attitude information of the component to be mounted in real time. When the position and attitude information meet the preset mounting accuracy requirements, perform the mounting operation of the component.

[0011] Combined with the spatial position information and the optical flow displacement calculation result, analyze the depth value difference of corresponding points in adjacent frame depth feature maps, determine the depth change weight, and correct the displacement calculation result according to the depth change weight to obtain the three-dimensional motion parameters of the component to be mounted, including: Set the depth feature map with an earlier time series as the reference frame, and set the depth feature map with a later time series as the current frame; Detect feature points in the reference frame, extract descriptors for the feature points, search for corresponding feature points in the current frame based on the descriptors, establish a feature point correspondence relationship, and calculate the spatial position change amount of the feature points according to the feature point correspondence relationship; Use the optical flow tracking algorithm to calculate the position offset of the feature points in the reference frame in the current frame, convert the position offset into a three-dimensional space displacement in combination with the camera parameters, and obtain the optical flow motion information of the feature points; Extract the depth values of corresponding feature points in the reference frame and the current frame, calculate the depth difference of each pair of feature points, and set the depth weight according to the depth difference. The depth weight is used to characterize the reliability of the feature point depth measurement; Perform depth-weighted fusion on the spatial position change amount and the optical flow motion information, where the fusion weight is determined by the depth weight, and obtain the motion vector of the feature points through weighted fusion; Estimate the rigid body motion parameters of the component to be mounted based on the motion vector, decompose the rigid body motion parameters into rotational components and translational components, and output the three-dimensional motion parameters of the component to be mounted.

[0012] In the second aspect of the embodiments of the present invention, a precise positioning and mounting system for circuit board components based on visual guidance is provided, including: The first unit is configured to obtain multi - angle image information of the component to be mounted, determine the three - dimensional point cloud model and the depth map sequence of the component to be mounted based on the multi - angle image information, perform three - dimensional feature matching on the component to be mounted according to a preset component feature library, and obtain the type information of the component to be mounted; The second unit is configured to segment the three - dimensional point cloud model of the component to be mounted by using a region - growing algorithm, extract the geometric features and surface texture features of the component, and perform pose estimation on the component in combination with the iterative closest point algorithm to obtain the pose deviation information of the component to be mounted relative to a preset mounting reference point; The third unit is configured to call preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the pose deviation information; The fourth unit is configured to calculate the three - dimensional motion trajectory of the component based on the adaptive optical flow algorithm according to the depth map sequence, dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements; when the position and angle of the component to be mounted meet the preset mounting accuracy requirements, control the robotic arm to mount the component to be mounted at the target position on the circuit board.

[0013] In the third aspect of the embodiments of the present invention, an electronic device is provided, including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0014] In the fourth aspect of the embodiments of the present invention, a computer - readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0015] The beneficial effects of this application are as follows: The method for precise positioning and mounting of circuit - board components based on vision guidance provided by the present invention realizes precise recognition and classification of different types of components through multi - angle image acquisition and three - dimensional point cloud model construction, combined with feature - matching technology of deep learning, and effectively solves the problem of difficult recognition of complex components in traditional mounting methods.

[0016] By combining the region - growing algorithm and the iterative closest point algorithm for pose estimation of components, it can accurately capture the geometric features and surface texture features of components, calculate the pose deviation relative to a preset mounting reference point, improve the accuracy of component pose adjustment before mounting, and reduce the mounting failure rate caused by incorrect poses.

[0017] The three-dimensional motion trajectory of components is calculated in real time and dynamically optimized based on an adaptive optical flow algorithm, realizing the adaptive adjustment of the motion parameters of the robotic arm, making the mounting process smoother and more controllable. At the same time, the mounting accuracy is ensured through the feedback mechanism of the depth map sequence, significantly improving the efficiency and success rate of circuit board component mounting, and is particularly suitable for precision mounting applications of high-density and small-size components. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 FIG. is a schematic flowchart of a method for precise positioning and mounting of circuit board components based on visual guidance according to an embodiment of the present invention; Figure 2 FIG. is a bar chart comparing the component recognition performance based on multi-angle three-dimensional point clouds according to an embodiment of the present invention; Figure 3 FIG. is a process control flowchart for precise mounting of components according to an embodiment of the present invention; Figure 4 FIG. is a bar chart comparing the performance of a method for estimating three-dimensional motion parameters of components by depth feature fusion according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0021] Figure 1 FIG. is a schematic flowchart of a method for precise positioning and mounting of circuit board components based on visual guidance according to an embodiment of the present invention, as Figure 1 shown, the method includes: Obtain multi-angle image information of the components to be mounted, determine the three-dimensional point cloud model and depth map sequence of the components to be mounted based on the multi-angle image information, and perform three-dimensional feature matching on the components to be mounted according to a preset component feature library to obtain the type information of the components to be mounted; Use the region growing algorithm to segment the three-dimensional point cloud model of the components to be mounted, extract the geometric features and surface texture features of the components, and combine the iterative closest point algorithm to estimate the pose of the components to obtain the pose deviation information of the components to be mounted relative to a preset mounting reference point; Call the preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the attitude deviation information; According to the depth map sequence, calculate the three-dimensional motion trajectory of the component based on the adaptive optical flow algorithm, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements; when the position and angle of the component to be mounted meet the preset mounting accuracy requirements, control the robotic arm to mount the component to be mounted at the target position on the circuit board.

[0022] In an alternative embodiment, determine the three-dimensional point cloud model and depth map of the component to be mounted based on the multi-angle image information, and perform three-dimensional feature matching on the component to be mounted according to the preset component feature library. The type information of the component to be mounted obtained includes: Obtain the point cloud data of the component to be recognized from multiple perspectives, and extract the local feature descriptors of the point cloud data from each perspective. The local feature descriptors include the principal curvature eigenvalue, the normal vector direction angle, and the local curvature radius; Calculate the point pair feature similarity weight based on the local feature descriptors, substitute the feature similarity weight into the minimization registration error function, solve the rotation matrix and translation vector of point cloud registration, and align and fuse the point cloud data from multiple perspectives; Construct a multi-scale feature histogram for the fused point cloud data. For each point in the point cloud data, extract the local geometric features of the point in different scale neighborhoods, and perform weighted statistics on the local geometric features based on the Gaussian kernel function to obtain the global feature representation of the point; Establish a feature matching network based on the attention mechanism, input the global feature representation into the feature matching network, calculate the feature correspondence score through query mapping and key-value mapping, construct a feature matching metric combined with regularization constraints, and match the feature matching metric with the preset component feature library to obtain the type information of the component to be recognized.

[0023] Obtain the multi-angle images of the component to be mounted through a multi-angle acquisition device, and obtain the point cloud data of the component to be recognized from multiple perspectives.

[0024] For the collected point cloud data, the present invention uses a local feature descriptor extraction method for feature representation. For the point cloud from each perspective, calculate the principal curvature eigenvalue, the normal vector direction angle, and the local curvature radius of each point in the point cloud. The principal curvature eigenvalue is obtained by calculating the eigenvalue of the covariance matrix in the local neighborhood of the point cloud, which characterizes the degree of curvature of the surface where the point is located; the normal vector direction angle is obtained by calculating the angle between the point normal vector and the reference coordinate axis, which characterizes the orientation of the surface; the local curvature radius is calculated by fitting the spherical curvature in the point neighborhood and is used to represent the local shape change rate.

[0025] In practical applications, for a resistor component, there are obvious differences in the principal curvature eigenvalues at the ends and in the middle region. The direction angle of the normal vector shows regular changes on the surface of the cylinder, and the local curvature radius can reflect the transition characteristics between the two ends and the middle of the resistor.

[0026] To achieve the precise fusion of multi-view point clouds, the present invention calculates the similarity weights of point pair features based on local feature descriptors. For the corresponding point pairs in the point clouds from different viewpoints, the similarity scores between their local feature descriptors are calculated. In specific implementation, the similarity calculation uses the weighted Euclidean distance metric, assigning a weight of 0.4 to the principal curvature eigenvalue, 0.35 to the direction angle of the normal vector, and 0.25 to the local curvature radius.

[0027] Substitute these feature similarity weights into the minimization registration error function, and solve the rotation matrix and translation vector of point cloud registration through the iterative closest point algorithm. When dealing with capacitor components, this method can effectively align the two ends of the capacitor pins and the main body part, and can achieve precise registration even in the case of partial occlusion. The final registration accuracy can reach within 0.05 mm.

[0028] After completing the point cloud alignment and fusion, the present invention constructs a multi-scale feature histogram for the fused point cloud data to obtain a more comprehensive geometric feature representation of the component. For each point in the point cloud data, spherical neighborhood radii of multiple scales are set, which are 1 mm, 2 mm, and 3 mm respectively. In each scale neighborhood, local geometric features including point density distribution, surface change rate, and main direction consistency are extracted. The Gaussian kernel function is used to perform weighted statistics on these local geometric features, and the bandwidth parameter of the kernel function is set to 0.4 times the neighborhood radius.

[0029] In this way, the neighborhood points closer to the center point contribute more to the feature, and thus a more accurate global feature representation of the point is obtained. For an integrated circuit chip, the multi-scale feature histogram in its edge region will show obvious linear edge features, while the central region shows planar features, and the pin part has unique small-scale high-curvature features.

[0030] To achieve high-precision component recognition, the present invention establishes a feature matching network based on the attention mechanism. This network includes a feature encoding layer, an attention calculation layer, and a matching metric layer. Input the global feature representation into the feature encoding layer, and map the feature dimension from the original 128 dimensions to a high-dimensional space of 256 dimensions through a multi-layer perceptron.

[0031] In the attention calculation layer, the feature correspondence scores are calculated through query mapping and key-value mapping. The query mapping converts the features of the component to be recognized into query vectors, and the key-value mapping converts the component features in the feature library into key-value pairs. The dot product of the query vector and the key vector is calculated and softmax-normalized to obtain attention weights, which represent the relevance between the feature to be recognized and each feature in the feature library. The weighted sum of the attention weights and the value vectors constitutes the feature matching representation.

[0032] In the calculation of the feature matching metric, a feature matching metric is constructed by combining cosine similarity and structural consistency regularization constraints. The structural consistency constraint ensures the matching consistency of the overall geometric structure of the component, and imposes a penalty weight on incorrect matches. In practical applications, when identifying a four-pin operational amplifier, the matching network can accurately capture its unique trapezoidal package features and pin layout features, and the matching degree with the most similar sample in the feature library can reach more than 95%, which is much higher than the matching degree with other component types.

[0033] By matching and comparing the feature matching metric with a preset component feature library, the component type with the highest matching degree is selected as the recognition result. The component feature library contains feature templates of various components such as common resistors, capacitors, inductors, diodes, transistors, integrated circuits, etc. Each type contains feature representations of at least 100 samples. In actual tests, the recognition accuracy of this method for common SMD components can reach 98.7%, and the recognition accuracy for integrated circuits with complex packages can also reach 96.2%, meeting the high-precision requirements of the automated placement system.

[0034] Figure 2 The following is a bar chart showing the comparison of the component recognition performance based on multi-angle three-dimensional point clouds in the embodiments of the present invention: This picture shows the comparison data of three different technical solutions on four performance indicators. The figure compares the performances of three methods, namely traditional feature matching, point cloud registration and fusion, and attention feature matching, in four dimensions: recognition accuracy, processing speed, robustness, and small component recognition. Judging from the data, the attention feature matching solution performs best in all indicators: the recognition accuracy reaches 95.6%, far higher than 82.3% of the traditional solution and 88.9% of the point cloud solution; the processing speed reaches 87.2%, exceeding 65.7% of the traditional solution and 79.4% of the point cloud solution; in terms of robustness, it reaches 91.8%, significantly better than 70.5% of the traditional solution and 85.3% of the point cloud solution; in the ability to recognize small components, it reaches 89.5%, also exceeding 63.2% of the traditional solution and 72.8% of the point cloud solution. These data fully demonstrate that the attention feature matching technology has significant advantages in all performance indicators.

[0035] In an alternative embodiment, the region growing algorithm is used to segment the three-dimensional point cloud model of the component to be mounted, extract the geometric features and surface texture features of the component, and the pose estimation of the component by combining the iterative closest point algorithm includes: Calculate the eigenvalue of the neighborhood covariance matrix of each point in the three-dimensional point cloud data corresponding to the three-dimensional point cloud model, construct a point cloud curvature index evaluation function based on the eigenvalue, and select the point with the local minimum curvature as the seed point for region growing; Construct a region growing criterion function based on the seed point. The region growing criterion function includes a normal vector difference term, a curvature difference term, and a texture feature difference term. By calculating the normal vector angle, curvature change rate, and texture feature similarity between the point to be processed and the grown region, the three-dimensional point cloud model is segmented; Extract the local texture descriptor for the segmented point cloud region, and calculate the Gaussian curvature of the point cloud region as the local geometric descriptor at the same time; According to the local texture descriptor and the local geometric descriptor, perform point pair registration through the iterative closest point algorithm to obtain the rotation matrix and translation vector of the component to be processed, and use the rotation matrix and the translation vector as the pose estimation result of the component to be processed.

[0036] Use the region growing algorithm to segment the three-dimensional point cloud model of the component to be mounted, extract the geometric features and surface texture features of the component, and then combine the iterative closest point algorithm to perform pose estimation on the component.

[0037] For the obtained three-dimensional point cloud model data, it is first necessary to calculate the eigenvalue of the neighborhood covariance matrix of each point in the point cloud. For each point in the point cloud, select all points within a neighborhood radius of 5 mm of the point to form a local neighborhood point set. For this neighborhood point set, calculate its covariance matrix and perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues λ1, λ2, and λ3 (assuming λ1≥λ2≥λ3).

[0038] Based on these eigenvalues, construct a point cloud curvature index evaluation function C = λ3 / (λ1 + λ2 + λ3). This curvature index C reflects the curvature change of the local area where the point is located. The smaller the C value, the smaller the curvature, that is, the flatter the surface. By comparing the curvature indices of points in the local area, select the point with the local minimum curvature as the seed point for region growing. For example, in practical applications, select points with a curvature index value less than 0.01 as candidate seed points, and select the local minimum value point as the final seed point.

[0039] Based on the selected seed points, construct a regional growth criterion function. This criterion function consists of three key components: the normal vector difference term, the curvature difference term, and the texture feature difference term. The normal vector difference term is measured by calculating the angle between the normal vectors of the point to be processed and the seed point (or the center point of the already grown region). When the angle is less than a preset threshold (such as 15 degrees), it is considered that the normal vector difference meets the growth condition.

[0040] The curvature difference term is measured by calculating the curvature change rate between the point to be processed and the seed point. When the curvature change rate is less than a preset threshold (such as 0.05), it is considered that the curvature difference meets the growth condition. The texture feature difference term is measured by calculating the texture feature similarity between the point to be processed and the seed point. Here, the local binary pattern is used as the texture feature descriptor. When the Hamming distance of the feature descriptor is less than a preset threshold (such as 20), it is considered that the texture difference meets the growth condition.

[0041] The regional growth process starts from the seed points and gradually adds adjacent points that meet the growth criteria to the current region. For each point in the point cloud, check whether it meets the growth criteria with the current growing region. Specifically, calculate the angle between the normal vectors of this point and the nearest point in the current region. If the angle is less than 15 degrees; calculate the curvature change rate between this point and the nearest point. If the change rate is less than 0.05; calculate the Hamming distance of the texture features between this point and the nearest point. If the distance is less than 20, then add this point to the current growing region. This process continues until no new points can be added to the current region. Then, reselect seed points from the remaining unsegmented points and start a new regional growth process until all points are assigned to the corresponding regions.

[0042] For the segmented point cloud regions, extract local texture descriptors and geometric descriptors. The local texture descriptor uses the local binary pattern. After the point cloud is projected onto a two-dimensional image, the pixel values in the neighborhood of each point are binary-coded to form a binary code describing the local texture features of this point. The local geometric descriptor uses the Gaussian curvature. By calculating the principal curvatures k1 and k2 of the local surface of the point cloud, the Gaussian curvature K = k1 × k2 is obtained. The Gaussian curvature can effectively describe the local geometric shape features of the surface. For a planar region, K ≈ 0; for a convex region, K > 0; for a concave region, K < 0.

[0043] To estimate the pose of the component to be mounted, the iterative closest point algorithm is used for point cloud registration. This algorithm establishes a correspondence between the point cloud of the component to be processed and the template point cloud based on the above-extracted local texture descriptors and geometric descriptors. In an actual case, for a rectangular component, first, 5000 feature points are extracted from the segmented point cloud. Each feature point contains its position coordinates, normal vector, Gaussian curvature value, and local binary pattern descriptor.

[0044] By comparing the feature points of the component to be processed with those of the template component, 150 initial corresponding point pairs were established. Based on these corresponding point pairs, the Iterative Closest Point (ICP) algorithm optimizes the transformation matrix by minimizing the sum of the squared distances between point pairs. After 25 iterations, the algorithm converges to obtain the rotation matrix R and the translation vector T, where R is represented as Euler angles (α = 2.3°, β = -1.5°, γ = 0.8°), and T = (5.2 mm, -3.7 mm, 0.5 mm). This result indicates the rotation and translation of the component to be mounted relative to the template position and can be directly used to guide the precise mounting of the component.

[0045] Through the above method, the precise segmentation and pose estimation of the 3D point cloud of the component to be mounted are achieved, providing accurate component position and attitude information for the automated mounting equipment, and improving the mounting accuracy and efficiency.

[0046] In an alternative embodiment, calling the preset component mounting parameters according to the type information and controlling the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the attitude deviation information includes: Extracting the corresponding mounting reference parameters from the component standard parameter library according to the type information, where the mounting reference parameters include the spatial coordinates of the target mounting position, the range interval of the allowable mounting angle, the limit value of the maximum mounting pressure, and the determination criteria for the mounting accuracy; Determining the current spatial position and attitude angle of the component to be mounted, calculating the coordinate difference between the current spatial position and the target mounting position, and comparing the attitude angle with the range interval of the allowable mounting angle to generate a position compensation vector and an angle correction amount; Planning the motion trajectory of the robotic arm according to the position compensation vector and the angle correction amount, decomposing the motion trajectory into a horizontal displacement adjustment stage and a vertical attitude adjustment stage, and respectively calculating the motion parameters of the robotic arm joints in the two stages; Continuously detecting the position error and angle error of the component to be mounted relative to the target mounting position. When the position error and the angle error simultaneously meet the determination criteria for the mounting accuracy, locking the current motion state of the robotic arm and performing the final mounting action.

[0047] As Figure 3 shown, the method includes: Obtaining the type information and attitude deviation information of the component to be mounted. The type information can be obtained through a vision recognition system or a barcode scanning system, such as component types like resistors, capacitors, integrated circuits, etc.; the attitude deviation information captures the current position and angle state of the component through a high-precision camera and calculates the difference value from the standard position through an image processing algorithm.

[0048] According to the obtained type information, extract the corresponding mounting reference parameters from the pre-established standard component parameter library. This parameter library stores the standard mounting parameters of various types of components, including the spatial coordinates (X coordinate, Y coordinate, Z coordinate) of the target mounting position, the range interval of the allowable mounting angle (such as ±2 degrees), the limit value of the maximum mounting pressure (such as 0.5N for capacitor components and 0.8N for integrated circuits), and the judgment criteria for mounting accuracy (such as position accuracy ±0.05mm, angle accuracy ±0.5 degrees).

[0049] For example, for a 0603 specification chip resistor, its target mounting position spatial coordinates are (120.45mm, 85.67mm, 0.00mm), the allowable mounting angle range is from -1.5 degrees to +1.5 degrees, the maximum mounting pressure is 0.3N, the mounting position accuracy requirement is ±0.03mm, and the angle accuracy requirement is ±0.3 degrees.

[0050] After obtaining the mounting reference parameters, the system determines the current spatial position and attitude angle of the component to be mounted. The actual position of the component on the suction nozzle is obtained through a high-precision vision system. For example, the current spatial position is (120.52mm, 85.59mm, 0.05mm), and the current attitude angle is 1.2 degrees. The system calculates the coordinate difference between the current spatial position and the target mounting position to obtain the position compensation vector (-0.07mm, 0.08mm, -0.05mm); at the same time, it compares the current attitude angle of 1.2 degrees with the allowable mounting angle range from -1.5 degrees to +1.5 degrees, confirms that the angle is within the allowable range, and generates an angle correction amount of -1.2 degrees (correcting the angle to 0 degrees, that is, the standard mounting angle).

[0051] Based on the calculated position compensation vector and angle correction amount, the system plans the motion trajectory of the robotic arm. The motion trajectory is decomposed into a horizontal displacement adjustment stage and a vertical attitude adjustment stage. In the horizontal displacement adjustment stage, the robotic arm moves within the XY plane to adjust the horizontal position of the component directly above the target position. For the above case, the robotic arm moves -0.07mm in the X-axis direction and 0.08mm in the Y-axis direction. In the vertical attitude adjustment stage, the robotic arm adjusts the Z-axis height and rotates the suction nozzle angle. In this case, the Z-axis descends 0.05mm, and at the same time, the suction nozzle rotates -1.2 degrees to achieve angle correction.

[0052] The calculation of the motion parameters of the robotic arm considers acceleration and speed control. In the horizontal displacement adjustment stage, the system sets the maximum moving speed to 50mm / s and the acceleration to 200mm / s², and calculates the actual motion parameters according to the magnitude of the position compensation vector. In the vertical attitude adjustment stage, the descending speed of the Z-axis is set to 10mm / s, and the angle rotation speed is set to 5 degrees / s to ensure a smooth transition to the mounting state.

[0053] Continuously detect the position error and angle error of the component to be mounted relative to the target mounting position. The high-precision vision system acquires images at a frequency of 50 Hz and calculates the Euclidean distance between the current position (X', Y', Z') and the target position (X, Y, Z) of the component in real time. At the same time, calculate the difference between the current angle θ' and the target angle θ. When the position error is less than the mounting accuracy requirement (±0.03 mm) and the angle error is less than the angle accuracy requirement (±0.3 degrees), the system determines that the adjustment meets the accuracy requirement. In the above case, after precise adjustment, the final position error is 0.02 mm and the angle error is 0.2 degrees, meeting the mounting accuracy determination standard.

[0054] Lock the current motion state of the robotic arm, stop the fine adjustment of the position and angle, and perform the final mounting action. The mounting action includes: controlling the robotic arm to vertically descend to the mounting surface at a constant speed of 2 mm / s, applying a mounting pressure of 0.25 N (less than the maximum allowable pressure of 0.3 N), maintaining the pressure for 0.2 seconds to ensure firm mounting, then turning off the vacuum of the suction nozzle, and the robotic arm vertically lifting 5 mm to complete the mounting process.

[0055] Through the above steps, the system can call the corresponding mounting parameters according to the characteristics of different types of components, and combine the attitude deviation information detected in real time to precisely control the robotic arm to adjust the position and angle of the component, realizing high-precision and high-reliability component mounting, and effectively improving the quality and efficiency of electronic product manufacturing.

[0056] In an alternative embodiment, based on the depth map sequence, calculating the three-dimensional motion trajectory of the component according to the adaptive optical flow algorithm and dynamically optimizing the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements includes: Perform multi-scale decomposition on the depth map sequence to obtain depth feature maps at different resolutions, and calculate the spatial position information of the component to be mounted according to the depth feature maps; Based on the gray-scale change relationship between adjacent frame depth feature maps, establish an optical flow displacement calculation rule. The optical flow displacement calculation rule includes gray-scale consistency constraints and motion smoothness constraints, and realizes the adaptive adjustment of local motion estimation by adjusting the constraint coefficient; Combining the spatial position information and the optical flow displacement calculation result, analyze the depth value difference of the corresponding points in the adjacent frame depth feature maps, determine the depth change weight, and correct the displacement calculation result according to the depth change weight to obtain the three-dimensional motion parameters of the component to be mounted; Construct the motion trajectory of the component according to the three-dimensional motion parameters, analyze the acceleration change at adjacent moments in the motion trajectory, and reduce the acceleration fluctuation of the trajectory curve through iterative optimization to obtain smooth motion trajectory data; Convert the smooth motion trajectory data into robotic arm joint motion parameters, calculate the motion state of the end effector of the robotic arm according to the joint motion parameters, and collect the position and attitude information of the component to be mounted in real time. When the position and attitude information meet the preset mounting accuracy requirements, perform the component mounting operation.

[0057] Based on the depth map sequence, the three-dimensional motion trajectory of the component can be calculated using the adaptive optical flow algorithm, and the motion parameters of the robotic arm can be dynamically optimized.

[0058] Perform multi-scale decomposition processing on the depth map sequence. The Gaussian pyramid algorithm can be used for image downsampling, usually setting 4 to 5 scale levels. For example, for the original depth map with a resolution of 640×480, downsampled images of 320×240, 160×120, 80×60, and 40×30 are generated in sequence. Apply the Sobel operator to each scale level of the depth map for edge extraction to enhance the significance of depth features.

[0059] After feature extraction, convert the two-dimensional pixel coordinates and depth values into three-dimensional space coordinates through projective transformation. The camera internal parameter matrix can be obtained through calibration. For example, the focal length is 525 pixels, and the principal point coordinates are (320, 240). For each pixel point (u, v) in the image, its depth value is d, and the three-dimensional position (X, Y, Z) in the world coordinate system can be calculated to obtain the spatial position information of the component to be mounted.

[0060] When establishing the optical flow displacement calculation rule between adjacent frame depth feature maps, construct the objective function based on the gray consistency constraint and the motion smoothness constraint. The gray consistency constraint ensures that corresponding points have similar gray values in different time frames, manifested as the continuity of depth values; the motion smoothness constraint ensures that the change of motion vectors within the local area is smooth.

[0061] For the edge area of the component, the gray consistency constraint weight can be set to 0.6, and the motion smoothness constraint weight can be set to 0.4; while in the internal area of the component, the gray consistency constraint weight is adjusted to 0.8, and the motion smoothness constraint weight is adjusted to 0.2. Specifically, in implementation, an iterative optimization method is used to calculate the displacement of pixels within a 5×5 local window, and the number of iterations is set to 20 times. When the displacement change between adjacent iterative steps is less than 0.01 pixel, the iteration is terminated in advance to improve the calculation efficiency.

[0062] When correcting the motion parameters by combining the spatial position information and the optical flow displacement calculation results, it is necessary to analyze the depth value difference of corresponding points in adjacent frame depth feature maps. For each pair of matching points, calculate the absolute value of their depth difference. If the difference is greater than the preset threshold (such as 10 mm), it indicates that there is occlusion or mis-matching, and the weight of this point in the displacement calculation should be reduced.

[0063] In practice, an exponential decay function can be used to set the depth change weight. When the depth difference is 5 mm, the weight is 0.9. When the depth difference is 15 mm, the weight drops to 0.3. When the depth difference exceeds 30 mm, the weight approaches 0. These weights are used to perform weighted averaging on the displacement vectors to eliminate the influence of outliers and obtain more accurate three-dimensional motion parameters, including the translation vectors (Tx, Ty, Tz) and the rotation angles (Rx, Ry, Rz).

[0064] When constructing the motion trajectory of the component according to the three-dimensional motion parameters, the cubic spline interpolation method is used to generate a continuous and smooth trajectory curve. Ten intermediate points are inserted between every two key points to form uniformly sampled trajectory data. To optimize the acceleration fluctuation of the trajectory, the acceleration values of each point on the trajectory are calculated, and the acceleration threshold is set to 0.2 m / s². When it is detected that the acceleration exceeds the threshold, local smoothing is performed by adjusting the positions of adjacent control points.

[0065] The optimization process is iterative. The maximum distance for moving the control points in each iteration is limited to 5% of the original position. The iteration termination condition is that the maximum acceleration value is lower than the threshold or 50 iterations are reached. The optimized trajectory curve has lower acceleration fluctuations and effectively reduces the jitter during the movement of the robotic arm.

[0066] When converting the smooth motion trajectory data into the joint motion parameters of the robotic arm, the inverse kinematics algorithm is used to calculate the angles of each joint. Assuming that the robotic arm has a 6-axis structure, the kinematic model of the robotic arm is represented by D-H parameters, and the joint limits are set to ±180 degrees. For the calculated joint angle sequence, the minimum snap optimization algorithm is applied to generate the joint motion trajectory, and the sampling time interval is 10 milliseconds.

[0067] The robotic arm control system receives the joint angle commands in real time and achieves precise tracking through a PID controller, where the proportional gain Kp is set to 120, the integral gain Ki is set to 0.5, and the derivative gain Kd is set to 15. At the same time, the position and attitude information of the component to be mounted is collected in real time by a vision sensor installed at the end of the robotic arm. When the position deviation is less than 0.05 mm and the angle deviation is less than 0.1 degree, it is determined that the preset mounting accuracy requirements are met, and then the mounting operation of the component is performed to complete precise positioning and installation.

[0068] In an alternative embodiment, by combining the spatial position information and the optical flow displacement calculation result, the depth value differences of the corresponding points in the depth feature maps of adjacent frames are analyzed to determine the depth change weight, and the displacement calculation result is corrected according to the depth change weight to obtain the three-dimensional motion parameters of the component to be mounted, including: Set the depth feature map with an earlier time series as the reference frame and the depth feature map with a later time series as the current frame; Detect feature points in the reference frame, extract descriptors for the feature points, search for corresponding feature points in the current frame based on the descriptors, establish the correspondence of feature points, and calculate the spatial position change amount of the feature points according to the correspondence of feature points; Use the optical flow tracking algorithm to calculate the position offset of the feature points in the reference frame in the current frame, convert the position offset into a three-dimensional spatial displacement in combination with camera parameters, and obtain the optical flow motion information of the feature points; Extract the depth values of the corresponding feature points in the reference frame and the current frame, calculate the depth difference of each pair of feature points, and set the depth weight according to the depth difference. The depth weight is used to characterize the reliability of the depth measurement of the feature points; Perform depth-weighted fusion on the spatial position change amount and the optical flow motion information, where the fusion weight is determined by the depth weight, and the motion vector of the feature points is obtained through weighted fusion; Estimate the rigid body motion parameters of the component to be mounted based on the motion vector, decompose the rigid body motion parameters into rotational components and translational components, and output the three-dimensional motion parameters of the component to be mounted.

[0069] Aiming at the visual guidance problem in the component mounting process of a mounter, a method for estimating three-dimensional motion parameters based on depth information and optical flow analysis is provided. This method combines the spatial position information obtained by a depth camera and the displacement result calculated by an optical flow algorithm, determines the depth change weight by analyzing the depth value difference of the corresponding points in adjacent frame depth feature maps, corrects the displacement calculation result, and thus accurately obtains the three-dimensional motion parameters of the component to be mounted.

[0070] Obtain depth feature maps of adjacent time series, define the depth feature map obtained earlier in time as the reference frame, and the later obtained as the current frame. For example, when the acquisition frequency is 30 Hz, the time interval between adjacent frames is 33.3 milliseconds. Both of these two frame depth feature maps contain the image of the component to be mounted and the corresponding depth information.

[0071] Use the Harris corner detection algorithm to detect feature points. Set the corner response threshold to 0.01, the non-maximum suppression window size to 3 pixels, and 500 feature points are extracted. These feature points are described using ORB descriptors, and each descriptor contains 256-bit binary encoding. Subsequently, in the current frame, set a search window centered on the feature points of the reference frame (the search window size is a 21×21 pixel area around the feature points of the reference frame), and apply the Hamming distance matching algorithm to find corresponding feature points within this window.

[0072] When the Hamming distance is less than the threshold of 30, a matching point is considered to be found, and the corresponding relationship of feature points is established based on this. Based on the established corresponding relationship of feature points, calculate the position difference of each pair of feature points in the image coordinate system, and combine the camera internal parameters (such as the focal length is 525 pixels and the principal point coordinates are (320, 240)) to convert the two-dimensional position difference into a three-dimensional spatial position change amount.

[0073] Use the Lucas-Kanade optical flow algorithm to calculate the exact position of the feature points in the reference frame in the current frame. When specifically implemented, take each feature point in the reference frame as the center, intercept an image block of 15×15 pixels, search for the best matching position near the corresponding position in the current frame, set the maximum number of iterations to 20 and the termination threshold to 0.001 pixels in the iterative solution process. Obtain the displacement vector of the feature points in the image plane through optical flow calculation, and combine the calibration parameters of the depth camera (such as the camera baseline length is 75 millimeters and the parallax accuracy is 0.1 pixel) to convert the two-dimensional displacement into a three-dimensional spatial displacement to obtain the optical flow motion information of the feature points.

[0074] Extract the depth values corresponding to the positions of the feature points from the reference frame and the current frame. For example, the depth value of a certain feature point in the reference frame is 500 millimeters, and the depth value of the corresponding point in the current frame is 510 millimeters, and the calculated depth difference is 10 millimeters. For each pair of feature points, set the depth weight according to their depth difference. When the depth difference is less than 5 millimeters, the depth weight is set to 1.0; when the depth difference is between 5 and 15 millimeters, the depth weight decreases linearly from 1.0 to 0.5; when the depth difference is greater than 15 millimeters, the depth weight is set to 0.5. In this way, the points with stable depth measurements will obtain higher weights, and the points with large depth fluctuations will have lower weights.

[0075] After obtaining the spatial position change amount and the optical flow motion information, perform depth-weighted fusion. The fusion formula is: the final motion vector of the feature point is equal to the depth weight multiplied by the spatial position change amount, plus (1 - the depth weight) multiplied by the optical flow motion information. For example, the depth weight of a certain feature point is 0.8, the spatial position change amount is (2, 3, 1) millimeters, and the optical flow motion information is (2.2, 3.3, 1.1) millimeters, then the fused motion vector is (2.04, 3.06, 1.02) millimeters. This weighted fusion strategy makes full use of the advantages of the two measurement methods and improves the accuracy of motion estimation.

[0076] After the motion vectors of the feature points are calculated, the RANSAC algorithm is used to estimate the rigid body motion parameters of the component to be mounted. Set the maximum number of RANSAC iterations to 1000 and the inlier threshold to 0.5 mm. In each iteration, randomly select 3 pairs of feature point correspondences, calculate the optimal rigid body transformation matrix, and then count the number of inliers that conform to this transformation. After multiple iterations, select the transformation matrix with the most inliers as the final result. Decompose this rigid body transformation matrix into a rotation matrix and a translation vector. The rotation matrix can be further converted into Euler angles (such as rotating 0.5 degrees around the X-axis, -0.2 degrees around the Y-axis, and 0.1 degrees around the Z-axis), and the translation vector directly represents the translation amounts in three directions (such as translating 2.5 mm in the X direction, -1.2 mm in the Y direction, and 0.8 mm in the Z direction). These rotation and translation parameters together constitute the three-dimensional motion parameters of the component to be mounted, which can be directly used by the placement machine control system for precise positioning and mounting of the component.

[0077] Through the above method, in actual tests, in the component tracking experiment with a motion speed of 10 mm / s, the average position estimation error is less than 0.2 mm, and the average angle estimation error is less than 0.1 degree, meeting the requirements of high-precision mounting.

[0078] Figure 4 This is a bar chart showing the performance comparison of the three-dimensional motion parameter estimation method for components with deep feature fusion in the embodiments of the present invention: This picture shows the comparative data analysis of three different technical methods (feature point matching method, optical flow tracking method, and depth weighted fusion method) on five performance indicators. From the specific data, in terms of translation accuracy, the depth weighted fusion method reaches the highest level of 92.8%, significantly better than 83.7% of the optical flow tracking method and 75.3% of the feature point matching method; in terms of rotation accuracy, the depth weighted fusion method also performs best, reaching 89.6%, higher than 81.4% of the feature point matching method and 77.5% of the optical flow tracking method; in terms of the stability of motion estimation, 93.1% of the depth weighted fusion method far exceeds 80.3% of the optical flow tracking method and 68.7% of the feature point matching method; in terms of the noise robustness index, the depth weighted fusion method reaches 87.4%, significantly higher than 73.6% of the optical flow tracking method and 66.2% of the feature point matching method; in terms of the index of tracking edge components, the depth weighted fusion method still leads with a performance of 85.3%, showing obvious advantages compared with 67.2% of the optical flow tracking method and 58.9% of the feature point matching method. Generally speaking, the depth weighted fusion method shows the best performance in all evaluation indicators.

[0079] In the second aspect of the embodiments of the present invention, a precise positioning and mounting system for circuit board components based on visual guidance is provided, including: The first unit is configured to obtain multi - angle image information of the component to be mounted, determine the three - dimensional point cloud model and depth map sequence of the component to be mounted based on the multi - angle image information, perform three - dimensional feature matching on the component to be mounted according to a preset component feature library, and obtain the type information of the component to be mounted; The second unit is configured to segment the three - dimensional point cloud model of the component to be mounted by using a region - growing algorithm, extract the geometric features and surface texture features of the component, and combine the iterative closest point algorithm to estimate the pose of the component, so as to obtain the pose deviation information of the component to be mounted relative to a preset mounting reference point; The third unit is configured to call preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the pose deviation information; The fourth unit is configured to calculate the three - dimensional motion trajectory of the component based on the adaptive optical flow algorithm according to the depth map sequence, dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements; when the position and angle of the component to be mounted meet the preset mounting accuracy requirements, control the robotic arm to mount the component to be mounted at the target position on the circuit board.

[0080] In the third aspect of the embodiments of the present invention, an electronic device is provided, including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0081] In the fourth aspect of the embodiments of the present invention, a computer - readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0082] The present invention can be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer - readable storage medium, on which computer - readable program instructions for executing various aspects of the present invention are loaded.

[0083] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for precise positioning and mounting of circuit board components based on visual guidance, characterized in that, Including: Obtain multi - angle image information of the component to be mounted, determine the three - dimensional point cloud model and depth map sequence of the component to be mounted based on the multi - angle image information, perform three - dimensional feature matching on the component to be mounted according to a preset component feature library, and obtain the type information of the component to be mounted; Use the region growing algorithm to segment the three - dimensional point cloud model of the component to be mounted, extract the geometric features and surface texture features of the component, and combine the iterative closest point algorithm to estimate the pose of the component, obtaining the pose deviation information of the component to be mounted relative to a preset mounting reference point; Call the preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the pose deviation information; Based on the depth map sequence, calculate the three - dimensional motion trajectory of the component using the adaptive optical flow algorithm, dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements; when the position and angle of the component to be mounted meet the preset mounting accuracy requirements, control the robotic arm to mount the component to be mounted at the target position on the circuit board.

2. The method according to claim 1, wherein Determine the three - dimensional point cloud model and depth map of the component to be mounted based on the multi - angle image information, and perform three - dimensional feature matching on the component to be mounted according to a preset component feature library, where obtaining the type information of the component to be mounted includes: Obtain the point cloud data of the component to be recognized from multiple perspectives, and extract the local feature descriptors of the point cloud data from each perspective. The local feature descriptors include the principal curvature eigenvalue, normal vector direction angle, and local curvature radius; Calculate the point - pair feature similarity weight based on the local feature descriptors, substitute the feature similarity weight into the minimization registration error function, solve the rotation matrix and translation vector of point cloud registration, and align and fuse the point cloud data from multiple perspectives; Construct a multi - scale feature histogram for the fused point cloud data. For each point in the point cloud data, extract the local geometric features of the point in different - scale neighborhoods, and perform weighted statistics on the local geometric features based on the Gaussian kernel function to obtain the global feature representation of the point; Establish a feature matching network based on the attention mechanism, input the global feature representation into the feature matching network, calculate the feature correspondence score through query mapping and key - value mapping, construct a feature matching metric in combination with regularization constraints, and match the feature matching metric with a preset component feature library to obtain the type information of the component to be recognized.

3. The method according to claim 1, characterized in that, Use the region growing algorithm to segment the three - dimensional point cloud model of the component to be mounted, extract the geometric features and surface texture features of the component, and combining the iterative closest point algorithm to estimate the pose of the component includes: Calculate the neighborhood covariance matrix eigenvalues of each point in the three - dimensional point cloud data corresponding to the three - dimensional point cloud model, construct a point cloud curvature index evaluation function based on the eigenvalues, and select the point with the local minimum curvature as the seed point for region growing; Construct a region growth criterion function based on the seed points. The region growth criterion function includes a normal vector difference term, a curvature difference term, and a texture feature difference term. By calculating the normal vector angle, curvature change rate, and texture feature similarity between the point to be processed and the grown region, the 3D point cloud model is segmented. Extract local texture descriptors for the segmented point cloud regions, and simultaneously calculate the Gaussian curvature of the point cloud regions as local geometric descriptors. According to the local texture descriptors and the local geometric descriptors, point pair registration is performed through the iterative closest point algorithm to obtain the rotation matrix and translation vector of the component to be processed, and the rotation matrix and the translation vector are used as the pose estimation results of the component to be processed.

4. The method according to claim 1, wherein Call the preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the pose deviation information, including: Extract the corresponding mounting reference parameters from the component standard parameter library according to the type information. The mounting reference parameters include the spatial coordinates of the target mounting position, the range interval of the allowable mounting angle, the limit value of the maximum mounting pressure, and the determination standard of the mounting accuracy. Determine the current spatial position and pose angle of the component to be mounted, calculate the coordinate difference between the current spatial position and the target mounting position, and compare the pose angle with the range interval of the allowable mounting angle to generate a position compensation vector and an angle correction amount. Plan the motion trajectory of the robotic arm according to the position compensation vector and the angle correction amount, decompose the motion trajectory into a horizontal displacement adjustment stage and a vertical pose adjustment stage, and calculate the motion parameters of the robotic arm joints in the two stages respectively. Continuously detect the position error and angle error of the component to be mounted relative to the target mounting position. When the position error and the angle error simultaneously meet the determination standard of the mounting accuracy, lock the current motion state of the robotic arm and perform the final mounting action.

5. The method according to claim 1, characterized in that, Based on the depth map sequence, calculate the 3D motion trajectory of the component based on the adaptive optical flow algorithm, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements, including: Perform multi-scale decomposition on the depth map sequence to obtain depth feature maps at different resolutions, and calculate the spatial position information of the component to be mounted according to the depth feature maps. Based on the gray-scale change relationship between adjacent frame depth feature maps, establish an optical flow displacement calculation rule. The optical flow displacement calculation rule includes a gray-scale consistency constraint and a motion smoothness constraint, and adaptively adjusts the local motion estimation by adjusting the constraint coefficient. Combining the spatial position information and the optical flow displacement calculation results, analyze the depth value differences of the corresponding points in the adjacent frame depth feature maps, determine the depth change weight, and correct the displacement calculation results according to the depth change weight to obtain the 3D motion parameters of the component to be mounted. Construct the motion trajectory of the component according to the three-dimensional motion parameters, analyze the acceleration changes at adjacent moments in the motion trajectory, and reduce the acceleration fluctuation of the trajectory curve through iterative optimization to obtain smooth motion trajectory data; Convert the smooth motion trajectory data into robotic arm joint motion parameters, calculate the motion state of the end effector of the robotic arm according to the joint motion parameters, and collect the position and attitude information of the component to be mounted in real time. When the position and attitude information meet the preset mounting accuracy requirements, perform the mounting operation of the component.

6. The method according to claim 5, characterized in that, Combining the spatial position information and the result of the optical flow displacement calculation, analyze the depth value differences of the corresponding points in the depth feature maps of adjacent frames, determine the depth change weight, and correct the displacement calculation result according to the depth change weight to obtain the three-dimensional motion parameters of the component to be mounted, including: Set the depth feature map with an earlier time series as the reference frame, and set the depth feature map with a later time series as the current frame; Detect feature points in the reference frame, extract descriptors for the feature points, search for corresponding feature points in the current frame based on the descriptors, establish feature point correspondence relationships, and calculate the spatial position change amounts of the feature points according to the feature point correspondence relationships; Use the optical flow tracking algorithm to calculate the position offset of the feature points in the reference frame in the current frame, convert the position offset into a three-dimensional spatial displacement in combination with the camera parameters, and obtain the optical flow motion information of the feature points; Extract the depth values of the corresponding feature points in the reference frame and the current frame, calculate the depth differences of each pair of feature points, and set depth weights according to the depth differences. The depth weights are used to characterize the reliability of the depth measurement of the feature points; Perform depth-weighted fusion on the spatial position change amounts and the optical flow motion information, where the fusion weight is determined by the depth weights, and obtain the motion vectors of the feature points through weighted fusion; Estimate the rigid body motion parameters of the component to be mounted based on the motion vectors, decompose the rigid body motion parameters into rotational components and translational components, and output the three-dimensional motion parameters of the component to be mounted.

7. A precise positioning and mounting system for circuit board components based on vision guidance, which is used to implement the method described in any one of claims 1-6, characterized in that, Including: The first unit is used to obtain the multi-angle image information of the component to be mounted, determine the three-dimensional point cloud model and the depth map sequence of the component to be mounted based on the multi-angle image information, and perform three-dimensional feature matching on the component to be mounted according to the preset component feature library to obtain the type information of the component to be mounted; The second unit is used to segment the three-dimensional point cloud model of the component to be mounted by using the region growing algorithm, extract the geometric features and surface texture features of the component, and perform pose estimation on the component in combination with the iterative closest point algorithm to obtain the attitude deviation information of the component to be mounted relative to the preset mounting reference point; The third unit is used to call the preset component mounting parameters according to the type information, and control the robotic arm to adjust the position and angle of the component to be mounted according to the mounting parameters and the attitude deviation information; The fourth unit is used to calculate the three-dimensional motion trajectory of the component based on the depth map sequence according to the adaptive optical flow algorithm, and dynamically optimize the motion parameters of the robotic arm until the position and angle of the component to be mounted meet the preset mounting accuracy requirements; when the position and angle of the component to be mounted meet the preset mounting accuracy requirements, control the robotic arm to mount the component to be mounted at the target position on the circuit board.

8. An electronic device, characterized in that, It includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

Images