Automatic assembling method and system for automobile safety belt pretensioner based on machine vision

Abstract

The invention provides an automatic assembling method and system for an automobile safety belt pretensioner based on machine vision, and relates to the technical field of automatic assembly.The method comprises the steps that firstly, images of standard parts of the pretensioner are collected, geometrical characteristic parameters are extracted, and the mapping relation from two-dimensional image coordinates to three-dimensional space coordinates is established; calculating coordinates of an assembly reference point and an assembly path point; acquiring a real-time image of a to-be-assembled part, extracting feature data, matching the feature data with a standard part, calculating a position deviation value and an angle deviation value, and updating an assembly path point coordinate; and controlling the robot to execute assembly, and dynamically adjusting the assembly speed and the assembly strength of the robot based on the assembly state evaluation index after data fusion of the force sensor and the displacement sensor until automatic assembly is completed.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic assembly, and in particular to a method and system for automatic assembly of a vehicle seat belt pretensioner based on machine vision. Background Art

[0002] Automobile seat belt pretensioners are key components in the vehicle's passive safety system, and their assembly quality is directly related to the reliability of the seat belt pretensioner function. With the rapid development of the automobile industry and the promotion and application of intelligent manufacturing technology, automatic assembly of pretensioners has become an important development direction for improving production efficiency and assembly quality. As an important support for industrial automation, machine vision technology can provide accurate position information and real-time status monitoring for automatic assembly of pretensioners.

[0003] In the prior art, there are still problems with the automatic assembly method of pretensioners. The traditional visual positioning method is difficult to adapt to the complex structural characteristics of pretensioner parts, especially in feature recognition and position determination under different viewing angles, where there are large errors. The position deviation compensation in the assembly process lacks real-time and accuracy, and often requires multiple adjustments to complete precise assembly. The assembly process lacks effective status monitoring and adaptive control mechanisms, which can easily lead to assembly failure or component damage due to improper assembly force or excessive position deviation.

[0004] In summary, there is an urgent need for an automatic assembly method for automobile seat belt pretensioners based on machine vision to improve the accuracy and reliability of automatic assembly of pretensioners, solve the technical problems of low positioning accuracy, poor real-time performance, and insufficient reliability in the prior art, and provide a practical and feasible technical solution for automatic assembly of pretensioners. The present invention can solve the problems in the prior art. Summary of the invention

[0005] The embodiment of the present invention provides a method and system for automatically assembling a vehicle seat belt pretensioner based on machine vision, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention, Provided is a method for automatically assembling a vehicle seat belt pretensioner based on machine vision, comprising: Collect component images of pretensioner standard parts at different angles, perform grayscale processing and Gaussian filtering noise reduction on the component images, use edge detection operators to extract edge features of the component images, segment the target area of ​​the edge features by region growing method, calculate and extract geometric feature parameters of the target area, project the geometric feature parameters to the world coordinate system, establish a mapping relationship from two-dimensional image coordinates to three-dimensional space coordinates, and calculate the assembly reference point and assembly path point coordinates of the pretensioner standard parts based on the mapping relationship; Collecting real-time images of parts to be assembled, performing adaptive histogram equalization enhancement processing on the real-time images, extracting feature data of the real-time images through a feature point detection algorithm; matching the feature data with geometric feature parameters of standard parts, obtaining reserved matching points through a projective transformation feature matching method, calculating position deviation values ​​and angle deviation values ​​of the parts to be assembled relative to an assembly reference point based on the reserved matching points, calculating compensation amounts according to the position deviation values ​​and angle deviation values, and updating the coordinates of assembly path points; The robot is controlled to perform component assembly according to the updated assembly path point coordinates, and the force sensor data and displacement sensor data during the assembly process are collected and input into the Kalman filter for data fusion to calculate the assembly state evaluation index, including the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve; when the assembly state evaluation index exceeds the preset threshold, the assembly speed and assembly force of the robot are dynamically adjusted according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed.

[0007] In an optional embodiment, Extracting edge features of the component image using an edge detection operator includes: Convolution operations are performed on the component image of the pretensioner standard component by using a first-order difference operator in the horizontal direction and a first-order difference operator in the vertical direction, respectively, to obtain a horizontal gradient value and a vertical gradient value of the component image; based on the horizontal gradient value and the vertical gradient value, a gradient amplitude and a gradient direction of each pixel point in the component image are calculated; Divide the component image into multiple local areas, calculate the gradient amplitude mean and the gradient amplitude standard deviation of each local area based on the gradient amplitude of each pixel point, calculate the local threshold of each local area according to the gradient amplitude mean and the gradient amplitude standard deviation, and use a bilinear interpolation method to smoothly transition the overlapping areas between the local areas; In each of the local areas, the gradient amplitude of each pixel is compared with the corresponding local threshold, and the gradient amplitude of the target pixel is compared with the gradient amplitude of the adjacent pixel points along the gradient direction corresponding to the target pixel point, and when the gradient amplitude of the target pixel point is greater than the local threshold and greater than the gradient amplitude of the adjacent pixel points, the target pixel point is marked as an edge point; Performing eight-neighborhood connectivity analysis on the edge point to obtain endpoint pixels, searching for connection points in a five-by-five neighborhood of the endpoint pixels, and connecting the two endpoint pixels when the distance between the two endpoint pixels is less than a preset distance threshold and the gradient direction difference is less than a preset angle threshold; The gradient amplitude of each edge point is interpolated along the gradient direction of each edge point, the edge point corresponding to the local maximum value is retained, the main edge point is determined, and the main edge point is processed by a morphological thinning algorithm to obtain an edge feature with a width of one pixel.

[0008] In an optional embodiment, Segmenting the target region of the edge feature by using a region growing method, and calculating and extracting geometric feature parameters of the target region include: Calculate the local variance of the component image where the edge feature is located, select pixel points whose grayscale values ​​are greater than a first preset grayscale threshold and are not edge points in the component image as candidate seed points, calculate the grayscale mean and grayscale standard deviation of the pixel points in the eight neighborhoods of the candidate seed points, and when the grayscale standard deviation is less than a second preset grayscale threshold and the difference between the grayscale value of the pixel points in the eight neighborhoods and the grayscale mean is less than a third preset grayscale threshold, determine the candidate seed points whose mutual spacing is greater than a preset distance threshold as growth seed points, and add the growth seed points to a queue to be processed; Based on each growth seed point in the queue to be processed, each pixel point in the eight neighborhoods of the growth seed point is marked as a point to be grown, and the grayscale difference between each point to be grown and the corresponding growth seed point is calculated. When the grayscale difference is less than the initial threshold of grayscale similarity, the average grayscale difference between the point to be grown and the grown area is calculated. When the average grayscale difference is less than the initial threshold of regional consistency and the point to be grown does not cross the edge feature, the point to be grown is added to the queue to be processed and marked as a processed pixel point; During the growth process, each time a preset number of pixel points are processed, the grayscale standard deviation of the grown area is calculated, and based on the ratio of the grayscale standard deviation to the grayscale standard deviation of the eight neighborhoods corresponding to the growth seed point, the grayscale similarity initial threshold and the regional consistency initial threshold are updated respectively to obtain a new grayscale similarity threshold and a new regional consistency threshold, and the growth process is iteratively performed; Based on the sub-regions formed by the growth of multiple growth seed points, the grayscale difference between adjacent sub-regions and the gradient strength of the sub-region boundaries are calculated. When the grayscale difference is less than the difference threshold and the gradient strength is less than the gradient strength threshold, the adjacent sub-regions are merged to obtain a merged sub-region; the merged sub-region is optimized to remove the noise region with an area less than the area threshold, fill the holes in the region, and smooth the region boundary to obtain the target region corresponding to the edge feature; the center of gravity coordinates, area value, perimeter value and roundness value of the target region are calculated to obtain the geometric feature parameters of the target region.

[0009] In an optional embodiment, Extracting feature data of the real-time image by a feature point detection algorithm includes: Perform multi-scale decomposition on the real-time image to construct a three-layer image pyramid, wherein the first layer image uses the original resolution, the second layer image downsamples the original resolution to one-half, and the third layer image downsamples the original resolution to one-quarter; Perform feature point detection in each layer of the three-layer image pyramid, obtain sixteen sampling points on a circle centered on the pixel point for each pixel point in each layer of the image, and calculate the grayscale difference between the pixel point and the sixteen sampling points; When there are nine or more consecutive sampling points among the sixteen sampling points whose grayscale differences are greater than a preset grayscale threshold, the pixel point is marked as a candidate feature point, the absolute value of the grayscale difference between the candidate feature point and the sixteen sampling points is calculated, and the maximum value of the difference between the absolute value of the grayscale difference and the preset grayscale threshold is taken as the response value of the candidate feature point; Perform non-maximum suppression on the candidate feature point, obtain pixel points in a three-by-three neighborhood with the candidate feature point as the center, calculate the response values ​​of all pixel points in the three-by-three neighborhood, and determine the candidate feature point as a feature point when the response value of the candidate feature point is greater than the response values ​​of all pixel points in the three-by-three neighborhood; Taking the feature point as the center, extract an image block of thirty-two pixels by thirty-two pixels, divide the image block into four by four sub-blocks, calculate the grayscale mean of each sub-block to obtain a sixteen-dimensional feature vector, compare each component in the sixteen-dimensional feature vector with other components to obtain binary bits, combine the binary bits to form a two-hundred-and-fifty-six-bit binary feature descriptor, and use the position information of the feature point and the binary feature descriptor as feature data of the real-time image.

[0010] In an optional embodiment, The feature data is matched with the geometric feature parameters of the standard parts, and the retained matching points obtained by the projective transformation feature matching method include: Perform feature matching based on the feature data and the geometric feature parameters, calculate the Hamming distance between the binary feature descriptor and the geometric feature parameters, find out the binary feature descriptor with the minimum Hamming distance and the second smallest Hamming distance for each geometric feature parameter, calculate the ratio of the minimum Hamming distance to the second smallest Hamming distance, and when the ratio is less than a preset threshold, determine the corresponding feature point position information and geometric feature parameters as a matching point pair; Randomly selecting three pairs of matching points from the matching point pairs, obtaining the coordinate position of each pair of matching points in the three pairs of matching points, constructing a coordinate mapping relationship according to the coordinate positions, establishing a linear equation group through the x-direction component and the y-direction component in the coordinate mapping relationship, solving the linear equation group to obtain transformation coefficients, and forming a transformation matrix for coordinate transformation with the transformation coefficients; Applying the transformation matrix to the position information of feature points in other matching point pairs except the three matching point pairs to obtain transformed position information, and calculating the Euclidean distance between the transformed position information and the geometric feature parameters in the corresponding matching point pairs as a reprojection error; Mark the matching point pairs whose reprojection errors are less than the error threshold as inliers that meet the transformation model, count the number of inliers of the inliers, compare the number of inliers with the current optimal number of inliers, and when the number of inliers is greater than the current optimal number of inliers, use the number of inliers to update the current optimal number of inliers, and use the corresponding transformation matrix to update the current optimal transformation matrix; The process is repeated until the preset maximum number of iterations is reached, and the inner point corresponding to the current optimal transformation matrix updated last is used as the reserved matching point.

[0011] In an optional embodiment, Control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation indicators, including the slope of the assembly force curve, the displacement change rate, and the force-displacement relationship curve. Control the robot to perform component assembly according to the coordinates of the assembly path points, and collect force data output by the force sensor and displacement data output by the displacement sensor on the end effector of the robot, wherein the force data includes force components and torque components in three directions, and the displacement data includes displacement components in three directions; Constructing a state vector including the force data, the displacement data, the force change rate and the displacement change rate, constructing an observation vector including the force data and the displacement data, calculating the deviation between the state vector and the predicted state vector at the previous moment to obtain a state error covariance matrix; calculating a theoretical covariance matrix and an actual covariance matrix based on the state error covariance matrix, calculating the ratio of the trace of the theoretical covariance matrix to the trace of the actual covariance matrix, and obtaining an adaptive factor; The product of the adaptive factor and the process noise covariance at the previous moment is used as the current process noise covariance, and the current measurement noise covariance is calculated based on the difference sequence of the state vector and the observation vector; the state vector, the observation vector, the process noise covariance and the measurement noise covariance are input into an adaptive Kalman filter, and a filtered state vector is obtained through a prediction step and an update step of the filter; Based on the force data in the filtered state vector, the ratio of the force difference to the time difference between adjacent sampling periods is calculated to obtain the slope of the assembly force curve. Based on the displacement data in the filtered state vector, the ratio of the displacement difference to the time difference between adjacent sampling periods is calculated to obtain the displacement change rate. Based on the ratio of the force difference to the displacement difference, a force-displacement relationship curve is obtained.

[0012] In an optional embodiment, When the assembly state evaluation index exceeds a preset threshold, dynamically adjusting the assembly speed and assembly force of the robot according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed includes: Inputting the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve into a fuzzy neural network, converting the input variables into fuzzy variables through the fuzzification layer of the fuzzy neural network, performing fuzzy reasoning at the rule layer according to a preset fuzzy rule library, and calculating the first force adjustment amount and the first speed adjustment amount at the defuzzification layer; The slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are used as the state space of reinforcement learning, and the force adjustment amount and the speed adjustment amount are used as the action space; the deviation of the slope of the assembly force curve from a first preset threshold, the deviation of the displacement change rate from a second preset threshold, and the deviation of the force-displacement relationship curve from a third preset threshold are calculated, and a reward value is calculated based on multiple deviations; the state space, the action space and the reward value are input into a deep Q network, and a second force adjustment amount and a second speed adjustment amount are obtained through iterative updating of the Q value; Calculating the fuzzy control weight and the reinforcement learning weight according to the assembly state evaluation index in the historical assembly cycle; adding the product of the first force adjustment amount and the fuzzy control weight to the product of the second force adjustment amount and the reinforcement learning weight to obtain the final force adjustment amount, and adding the product of the first speed adjustment amount and the fuzzy control weight to the product of the second speed adjustment amount and the reinforcement learning weight to obtain the final speed adjustment amount; The final force adjustment amount is added to the current assembly force to obtain an updated assembly force, and the final speed adjustment amount is added to the current assembly speed to obtain an updated assembly speed; until the updated slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are all within the corresponding preset threshold ranges, the automatic assembly of parts is completed.

[0013] According to a second aspect of the embodiments of the present invention, Provided is a vehicle seat belt pretensioner automatic assembly system based on machine vision, comprising: The first unit is used to collect component images of the pretensioner standard component at different angles, perform grayscale processing and Gaussian filtering noise reduction on the component image, extract edge features of the component image using an edge detection operator, segment the target area of ​​the edge feature by a region growing method, calculate and extract geometric feature parameters of the target area, project the geometric feature parameters to a world coordinate system, establish a mapping relationship from two-dimensional image coordinates to three-dimensional space coordinates, and calculate the assembly reference point and assembly path point coordinates of the pretensioner standard component based on the mapping relationship; The second unit is used to collect real-time images of parts to be assembled, perform adaptive histogram equalization enhancement processing on the real-time images, extract feature data of the real-time images through a feature point detection algorithm; match the feature data with geometric feature parameters of standard parts, obtain reserved matching points through a projective transformation feature matching method, calculate the position deviation value and angle deviation value of the parts to be assembled relative to the assembly reference point based on the reserved matching points, calculate the compensation amount according to the position deviation value and the angle deviation value, and update the coordinates of the assembly path points; The third unit is used to control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation index, including the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve; when the assembly state evaluation index exceeds the preset threshold, the assembly speed and assembly force of the robot are dynamically adjusted according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed.

[0014] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0016] In the embodiment of the present invention, by accurately identifying and locating the parts to be assembled through machine vision and compensating based on the deviation value, the assembly accuracy of the automobile seat belt pretensioner can be significantly improved, ensuring the consistency and stability of the assembly quality; by using a robot to automatically execute the assembly process and dynamically adjusting the assembly parameters according to the assembly status evaluation index, the assembly time can be effectively shortened and the production efficiency can be improved; by integrating technologies such as machine vision, robot control and sensor data processing, the automated and intelligent assembly of the automobile seat belt pretensioner is realized, the dependence on manual operation is reduced, and the level of automation in production is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of an automatic assembly method of a vehicle seat belt pretensioner based on machine vision according to an embodiment of the present invention; Figure 2 The present invention is a schematic structural diagram of an automatic assembly system for a vehicle seat belt pretensioner based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Figure 1 FIG. 1 is a flow chart of an automatic assembly method of a vehicle seat belt pretensioner based on machine vision according to an embodiment of the present invention. Figure 1 As shown, the method includes: S101. Collect component images of pretensioner standard components at different angles, perform grayscale processing and Gaussian filtering noise reduction on the component images, extract edge features of the component images using edge detection operators, segment the target region of the edge features using region growing method, calculate and extract geometric feature parameters of the target region, project the geometric feature parameters to the world coordinate system, establish a mapping relationship from two-dimensional image coordinates to three-dimensional space coordinates, and calculate the assembly reference point and assembly path point coordinates of the pretensioner standard components based on the mapping relationship; In an optional specific implementation, first, component images are collected at different viewing angles of the pretensioner standard components. Image data of the pretensioner standard components are collected by an industrial camera within a range of 0° to 360° at an angle interval of 5°. Grayscale processing is performed on the collected image to convert the RGB three-channel image into a single-channel grayscale image. Then, a Gaussian filter is used to perform noise reduction processing on the grayscale image, and the image is smoothed to remove noise by setting the size and standard deviation parameters of the Gaussian kernel.

[0021] Secondly, extract the edge features of the component image. Use the first-order difference operator in the horizontal and vertical directions to perform convolution operations with the image respectively to obtain the gradient values ​​in the horizontal and vertical directions. Calculate the gradient amplitude and gradient direction of each pixel based on the gradient values ​​in these two directions. Divide the image into multiple overlapping local areas, calculate the statistical characteristics of the gradient amplitude in each local area, and obtain the local adaptive threshold. Use bilinear interpolation to achieve smooth transition in the overlapping area. Compare the gradient amplitude of each pixel with the local threshold and the gradient amplitude of the adjacent pixels to determine the edge point. Perform eight-neighborhood connectivity analysis on the edge points, find the endpoints and connect them. Finally, the edge features with a single pixel width are obtained through gradient direction interpolation and morphological refinement.

[0022] Next, the target region is segmented using the region growing method. First, candidate seed points that meet the grayscale conditions are selected in the non-edge region, and the final growth seed points are determined by analyzing the grayscale distribution characteristics of their eight neighborhoods. These seed points are used as starting points, and region growing is performed based on the grayscale similarity and region consistency criteria. During the growth process, the similarity threshold and consistency threshold are dynamically adjusted to achieve adaptive growth. The sub-regions obtained by growth are merged and optimized, the noise region is removed, the holes are filled, and the boundaries are smoothed, and finally the complete target region is obtained.

[0023] Then, the geometric characteristic parameters of the target area are extracted. The coordinates of the center of gravity of the target area are calculated, the area value of the area is counted, the perimeter value of the area boundary is calculated, and the roundness value is calculated based on the area and perimeter. These characteristic parameters are used to characterize the location and shape characteristics of the target area.

[0024] Finally, the mapping relationship from 2D image coordinates to 3D space coordinates is established. Based on the pre-calibrated camera parameters, the projection transformation relationship between the image coordinate system and the world coordinate system is established. The geometric feature parameters at different angles are projected into the world coordinate system to construct the correspondence between the feature parameters and the spatial position. Based on this correspondence, the assembly reference point position of the pretensioner standard parts is determined, and a complete assembly path point coordinate sequence is planned according to the assembly process requirements.

[0025] In this embodiment, graying and Gaussian filtering processes remove noise, enhance image clarity, and contribute to the accuracy of edge detection; through edge detection and region growing methods, the edge features of the target area are accurately extracted, interference is avoided, and accurate segmentation of the target area is ensured; the geometric features of the target area are calculated, providing key data support for the subsequent assembly process; by mapping the image coordinates to three-dimensional space, accurate positioning of standard parts is achieved, ensuring accurate calculation of assembly reference points and path points; providing the robot with accurate assembly paths and reference points, improving assembly accuracy and efficiency, and reducing assembly errors.

[0026] S102. Collect real-time images of parts to be assembled, perform adaptive histogram equalization enhancement processing on the real-time images, extract feature data of the real-time images through feature point detection algorithm; match the feature data with geometric feature parameters of standard parts, obtain reserved matching points through projective transformation feature matching method, calculate position deviation value and angle deviation value of the parts to be assembled relative to the assembly reference point based on the reserved matching points, calculate compensation amount according to the position deviation value and angle deviation value, and update the coordinates of assembly path points; In an optional specific implementation, first, real-time images of the parts to be assembled are collected. An industrial camera is used to take real-time photos of the parts to be assembled to obtain image data in the current state. Adaptive histogram equalization is performed on the collected images to enhance image contrast and improve image detail features by performing histogram equalization on local areas of the image.

[0027] Then, the feature data of the real-time image is extracted. A three-layer image pyramid is constructed, and feature points are detected for each layer of the image. Candidate feature points are identified by analyzing the grayscale differences of the sampling points on the circumference around each pixel. Non-maximum suppression is performed on the candidate feature points, and the feature points with the strongest local response are retained. Local image blocks are extracted around the feature points, and the image blocks are divided into sub-regions and grayscale statistical features are calculated to generate binary feature descriptors.

[0028] Next, the feature matching process is performed. The Hamming distance between the real-time image feature descriptor and the standard component geometric feature parameters is calculated, and the initial matching point pairs are selected based on the distance ratio criterion. The random sampling consistency algorithm is used to calculate the projective transformation matrix by randomly selecting matching point pairs, evaluate the reprojection errors of other matching points, and iteratively optimize to obtain the optimal transformation matrix and retain matching points.

[0029] Then, the position deviation and angle deviation are calculated. Based on the spatial distribution of the retained matching points, the position offset and attitude angle difference of the parts to be assembled relative to the preset assembly reference point are calculated. The position deviation is calculated by the coordinate difference of the matching points, and the angle deviation is determined by the angle between the distribution directions of the feature points.

[0030] Finally, the assembly path points are updated. Based on the calculated position deviation and angle deviation values, combined with the assembly accuracy requirements, the displacement and rotation angle that need to be compensated are determined. The compensation amount is superimposed on the original assembly path point coordinates to obtain an updated path point sequence to guide the subsequent precise assembly process.

[0031] In this embodiment, the adaptive histogram equalization process enhances the image contrast, makes the image details clearer, and contributes to the accuracy of the subsequent feature extraction and matching process, especially under complex lighting conditions, which can effectively improve the image quality; by constructing an image pyramid and multi-layer feature point detection, rich image features can be extracted at different scales. Non-maximum suppression and the generation of binary descriptors make the extraction of feature points more accurate and stable, ensuring the feature point data with high reliability in the subsequent matching process; by using the projective transformation feature matching and RANSAC algorithm, the correct matching pairs can be screened out from a large number of matching points, and the optimal transformation matrix can be obtained through iterative optimization, which greatly improves the accuracy of feature matching, reduces the interference of mismatching, and ensures the accuracy of assembly path point calculation; by calculating the position deviation and angle deviation, the difference between the parts to be assembled and the preset reference point can be accurately evaluated, ensuring the precision control in the assembly process and effectively avoiding assembly errors; according to the deviation calculation results, the assembly path points are dynamically adjusted to compensate for the deviation value, ensuring the accuracy of the path in the robot assembly process, optimizing the assembly accuracy, reducing the assembly deviation caused by inaccurate positioning, and improving the overall assembly efficiency and accuracy.

[0032] S103. Control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation index, including the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve; when the assembly state evaluation index exceeds the preset threshold, dynamically adjust the assembly speed and assembly force of the robot according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed.

[0033] In this embodiment, by collecting data from force sensors and displacement sensors in real time, the force applied by the robot and the displacement changes generated during the assembly process can be accurately tracked. This real-time monitoring provides key feedback for subsequent adjustments, ensuring that problems in the assembly process can be discovered and corrected in a timely manner; the Kalman filter fuses sensor data to effectively remove noise and improve data accuracy. In this way, an accurate assessment of the assembly state can be obtained in real time, helping to determine whether the assembly meets expectations, while improving the robot system's ability to respond to abnormal situations; through real-time analysis of assembly force and displacement changes, when the system finds that the assembly state exceeds the preset threshold, it can dynamically adjust the robot's assembly speed and strength. This adaptive adjustment mechanism can avoid assembly accuracy deviations caused by excessive or too small force, too fast or too slow speed, thereby ensuring assembly quality; the dynamic adjustment mechanism ensures that the robot can adapt to various assembly requirements, whether it is force control or displacement adjustment, it can be accurately executed. The errors in the assembly process are effectively controlled, improving the overall assembly accuracy and consistency; the entire process is highly automated and intelligent. The robot no longer relies on manual intervention to adjust parameters, but automatically optimizes its own operations based on real-time data and feedback, greatly improving assembly efficiency and quality.

[0034] In an optional implementation, extracting edge features of the component image using an edge detection operator includes: Convolution operations are performed on the component image of the pretensioner standard component by using a first-order difference operator in the horizontal direction and a first-order difference operator in the vertical direction, respectively, to obtain a horizontal gradient value and a vertical gradient value of the component image; based on the horizontal gradient value and the vertical gradient value, a gradient amplitude and a gradient direction of each pixel point in the component image are calculated; Divide the component image into multiple local areas, calculate the gradient amplitude mean and the gradient amplitude standard deviation of each local area based on the gradient amplitude of each pixel point, calculate the local threshold of each local area according to the gradient amplitude mean and the gradient amplitude standard deviation, and use a bilinear interpolation method to smoothly transition the overlapping areas between the local areas; In each of the local areas, the gradient amplitude of each pixel is compared with the corresponding local threshold, and the gradient amplitude of the target pixel is compared with the gradient amplitude of the adjacent pixel points along the gradient direction corresponding to the target pixel point, and when the gradient amplitude of the target pixel point is greater than the local threshold and greater than the gradient amplitude of the adjacent pixel points, the target pixel point is marked as an edge point; Performing eight-neighborhood connectivity analysis on the edge point to obtain endpoint pixels, searching for connection points in a five-by-five neighborhood of the endpoint pixels, and connecting the two endpoint pixels when the distance between the two endpoint pixels is less than a preset distance threshold and the gradient direction difference is less than a preset angle threshold; The gradient amplitude of each edge point is interpolated along the gradient direction of each edge point, the edge point corresponding to the local maximum value is retained, the main edge point is determined, and the main edge point is processed by a morphological thinning algorithm to obtain an edge feature with a width of one pixel.

[0035] First, a grayscale image of the standard parts of the pretensioner is obtained. For example, a grayscale image of the pretensioner parts with a resolution of 1024×768 pixels is obtained by shooting with an industrial camera.

[0036] Then, the first-order difference operator in the horizontal and vertical directions is used to perform convolution operations on the acquired grayscale image. For example, [-1, 0, 1] is used as the first-order difference operator in the horizontal direction, and the transpose of [-1, 0, 1] is used as the first-order difference operator in the vertical direction, and the image is convolved to obtain the horizontal gradient value and the vertical gradient value of each pixel. For example, if the grayscale value of a pixel in the image and the grayscale value of the pixel to its right are 50 and 70 respectively, then the horizontal gradient value of the pixel is 70-50=20.

[0037] Next, the gradient magnitude and gradient direction of each pixel are calculated based on the horizontal gradient value and vertical gradient value of each pixel. The gradient magnitude can be understood as the size of the gradient, and the gradient direction indicates the direction in which the gradient changes fastest. For example, if the horizontal gradient value of a pixel is 20 and the vertical gradient value is 30, then the gradient magnitude of the pixel is about 36 (calculated using the Pythagorean theorem) and the gradient direction is about 56.3 degrees (calculated using the inverse tangent function).

[0038] After that, the image is divided into multiple local regions. For example, a 1024×768 image is divided into multiple local regions of 32×32 pixels. For each local region, the mean and standard deviation of the gradient amplitudes of all pixels in the region are calculated. For example, a local region contains 1024 pixels, and their gradient amplitudes are 10, 15, 20, ..., respectively. The mean gradient amplitude of the region is the sum of all gradient amplitudes divided by the number of pixels 1024. The standard deviation of the gradient amplitude reflects the dispersion of the gradient amplitude in the region.

[0039] The local threshold of each local region is calculated based on the mean and standard deviation of the gradient amplitude. For example, the local threshold is set to a multiple of the mean plus the standard deviation. This multiple can be adjusted according to the actual situation, for example, it is set to 1.5 times. For overlapping areas between local regions, the bilinear interpolation method is used to smoothly transition the local threshold to avoid threshold mutations. For example, for overlapping pixels in two adjacent regions, the local threshold is the weighted average of the thresholds of the two regions, and the weight is related to the distance from the pixel to the center of the two regions.

[0040] In each local area, the gradient magnitude of each pixel is compared with the corresponding local threshold. At the same time, the gradient magnitude of the pixel is compared with the gradient magnitude of its neighboring pixels along the gradient direction. If the gradient magnitude of the pixel is greater than both the local threshold and the gradient magnitude of its neighboring pixels, the pixel is marked as an edge point. For example, if the gradient magnitude of a pixel is 50, the local threshold is 40, and the gradient magnitude of its neighboring pixels along the gradient direction is 45, the pixel will be marked as an edge point.

[0041] Perform eight-neighborhood connectivity analysis on all pixels marked as edge points to find the endpoints of the edge. Search for other endpoints in the 5×5 neighborhood of each endpoint pixel. If the distance between the two endpoints is less than a preset distance threshold (for example, set to 10 pixels) and the gradient direction difference between the two endpoint pixels is less than a preset angle threshold (for example, set to 30 degrees), then connect the two endpoints.

[0042] The gradient amplitude of each edge point is interpolated along its gradient direction. The edge point corresponding to the local maximum is retained and determined as the main edge point. For example, if the gradient amplitude of an edge point is 50, and the gradient amplitudes of its two adjacent points along the gradient direction are 45 and 48 respectively, then the edge point is considered to be a local maximum point and is determined as the main edge point. Finally, a morphological thinning algorithm, such as a skeletonization algorithm, is used to process the main edge point to obtain an edge feature with a width of one pixel.

[0043] In this embodiment, by comparing the local threshold and gradient direction, noise and non-edge pixels can be effectively suppressed and the accuracy of edge detection can be improved; by endpoint connection and gradient direction interpolation, broken edges can be effectively connected and the continuity of the edges can be maintained, thereby better describing the contour information of parts; by using the morphological thinning algorithm, the edges can be thinned to a single pixel width, which is convenient for subsequent image processing and analysis, such as feature matching and recognition.

[0044] In an optional implementation, segmenting the target region of the edge feature by a region growing method, and calculating and extracting the geometric feature parameters of the target region includes: Calculate the local variance of the component image where the edge feature is located, select pixel points whose grayscale values ​​are greater than a first preset grayscale threshold and are not edge points in the component image as candidate seed points, calculate the grayscale mean and grayscale standard deviation of the pixel points in the eight neighborhoods of the candidate seed points, and when the grayscale standard deviation is less than a second preset grayscale threshold and the difference between the grayscale value of the pixel points in the eight neighborhoods and the grayscale mean is less than a third preset grayscale threshold, determine the candidate seed points whose mutual spacing is greater than a preset distance threshold as growth seed points, and add the growth seed points to a queue to be processed; Based on each growth seed point in the queue to be processed, each pixel point in the eight neighborhoods of the growth seed point is marked as a point to be grown, and the grayscale difference between each point to be grown and the corresponding growth seed point is calculated. When the grayscale difference is less than the initial threshold of grayscale similarity, the average grayscale difference between the point to be grown and the grown area is calculated. When the average grayscale difference is less than the initial threshold of regional consistency and the point to be grown does not cross the edge feature, the point to be grown is added to the queue to be processed and marked as a processed pixel point; During the growth process, each time a preset number of pixel points are processed, the grayscale standard deviation of the grown area is calculated, and based on the ratio of the grayscale standard deviation to the grayscale standard deviation of the eight neighborhoods corresponding to the growth seed point, the grayscale similarity initial threshold and the regional consistency initial threshold are updated respectively to obtain a new grayscale similarity threshold and a new regional consistency threshold, and the growth process is iteratively performed; Based on the sub-regions formed by the growth of multiple growth seed points, the grayscale difference between adjacent sub-regions and the gradient strength of the sub-region boundaries are calculated. When the grayscale difference is less than the difference threshold and the gradient strength is less than the gradient strength threshold, the adjacent sub-regions are merged to obtain a merged sub-region; the merged sub-region is optimized to remove the noise region with an area less than the area threshold, fill the holes in the region, and smooth the region boundary to obtain the target region corresponding to the edge feature; the center of gravity coordinates, area value, perimeter value and roundness value of the target region are calculated to obtain the geometric feature parameters of the target region.

[0045] The eight-neighborhood specifically refers to the eight adjacent pixels around a pixel point, including adjacent pixels in the horizontal, vertical and diagonal directions. Specifically, the eight-neighborhood includes the following eight pixels: pixels on the left and right of the horizontal neighborhood; pixels on the top and bottom of the vertical neighborhood; pixels at the upper left corner, upper right corner, lower left corner and lower right corner of the diagonal neighborhood.

[0046] In an optional specific implementation, first, obtain the component image to be processed and perform edge detection preprocessing on it, such as using the Canny operator to extract the image edge. Assume that the obtained component image is a 256×256 grayscale image, in which the grayscale value of the target area is generally higher than that of the background area.

[0047] Next, the local variance of the component image is calculated to reflect the texture information of the image. For example, a 3×3 window is used to calculate the local variance of each pixel. Regions with large local variances usually correspond to the edges of the image or regions with rich texture details.

[0048] Then, pixels whose grayscale values ​​are greater than the first preset grayscale threshold and are not edge points are selected as candidate seed points. For example, if the first preset grayscale threshold is set to 150, then pixels whose grayscale values ​​are greater than 150 and are not on the edge will be selected as candidate seed points. Assume that 50 pixels meet the conditions.

[0049] For each candidate seed point, calculate the grayscale mean and grayscale standard deviation of the pixels in its eight neighborhoods. For example, if the grayscale values ​​of the pixels in the eight neighborhoods of a candidate seed point are 160, 155, 162, 158, 165, 152, 161, and 159, then its grayscale mean is 159 and its grayscale standard deviation is 4.2.

[0050] Determine whether the candidate seed point meets the conditions for becoming a growth seed point: the grayscale standard deviation is less than the second preset grayscale threshold and the difference between the grayscale value of the pixel point in the eight neighborhoods and the grayscale mean is less than the third preset grayscale threshold. For example, the second preset grayscale threshold is set to 5 and the third preset grayscale threshold is set to 10. If these two conditions are met, the candidate seed point is preliminarily identified as a growth seed point.

[0051] In order to avoid the seed points being too dense, further screen the growth seed points. Calculate the candidate seed points whose distances are greater than the preset distance threshold, and finally determine the candidate seed points as growth seed points. For example, set the preset distance threshold to 5 pixels. Assume that 10 growth seed points are finally screened out. Add these growth seed points to the queue to be processed.

[0052] Next, the region growing process begins. A growth seed point is taken from the queue to be processed, and each pixel in its eight-neighborhood is marked as a point to be grown.

[0053] For each point to be grown, calculate the grayscale difference between it and the corresponding growth seed point. If the grayscale difference is less than the initial grayscale similarity threshold (for example, the initial value is 8), proceed to the next step of judgment.

[0054] Calculate the average grayscale difference between the point to be grown and the grown area. If the average grayscale difference is less than the initial threshold of regional consistency (for example, the initial value is 6) and the point to be grown does not cross the previously detected edge, then add the point to be grown to the queue to be processed and mark it as a processed pixel, and merge it into the growing area.

[0055] During the growth process, the grayscale standard deviation of the grown area is calculated every time a certain number of pixels are processed (for example, every time 50 pixels are processed). According to the ratio of the grayscale standard deviation to the grayscale standard deviation of the eight neighborhoods corresponding to the growth seed point, the grayscale similarity initial threshold and the regional consistency initial threshold are updated to obtain new thresholds. For example, if the ratio is greater than 1.2, the grayscale similarity threshold and the regional consistency threshold are multiplied by 1.1 respectively; if the ratio is less than 0.8, the grayscale similarity threshold and the regional consistency threshold are multiplied by 0.9 respectively. Then continue to iterate the growth process.

[0056] After multiple growth seed points grow, multiple sub-regions will be formed. For adjacent sub-regions, the grayscale difference between adjacent sub-regions and the gradient strength of the sub-region boundary are calculated. If the grayscale difference is less than the difference threshold (for example, set to 15) and the gradient strength is less than the gradient strength threshold (for example, set to 20), the two sub-regions are merged.

[0057] The merged sub-regions are optimized. The noise regions with an area smaller than the area threshold (for example, set to 50 pixels) are removed, the holes in the region are filled, and the region boundaries are smoothed to finally obtain the target region corresponding to the edge features.

[0058] Finally, the centroid coordinates, area value, perimeter value and roundness value of the target area are calculated to obtain the geometric characteristic parameters of the target area.

[0059] In this embodiment, by dynamically adjusting the growth threshold and merging sub-regions, the target area can be segmented more accurately, especially in the presence of noise and uneven lighting; it has less dependence on initial parameters and can adapt to different types of images and different application scenarios; it can calculate a variety of geometric feature parameters of the target area to provide more comprehensive information for subsequent image analysis and recognition.

[0060] In an optional implementation, extracting feature data of the real-time image by a feature point detection algorithm includes: Perform multi-scale decomposition on the real-time image to construct a three-layer image pyramid, wherein the first layer image uses the original resolution, the second layer image downsamples the original resolution to one-half, and the third layer image downsamples the original resolution to one-quarter; Perform feature point detection in each layer of the three-layer image pyramid, obtain sixteen sampling points on a circle centered on the pixel point for each pixel point in each layer of the image, and calculate the grayscale difference between the pixel point and the sixteen sampling points; When there are nine or more consecutive sampling points among the sixteen sampling points whose grayscale differences are greater than a preset grayscale threshold, the pixel point is marked as a candidate feature point, the absolute value of the grayscale difference between the candidate feature point and the sixteen sampling points is calculated, and the maximum value of the difference between the absolute value of the grayscale difference and the preset grayscale threshold is taken as the response value of the candidate feature point; Perform non-maximum suppression on the candidate feature point, obtain pixel points in a three-by-three neighborhood with the candidate feature point as the center, calculate the response values ​​of all pixel points in the three-by-three neighborhood, and determine the candidate feature point as a feature point when the response value of the candidate feature point is greater than the response values ​​of all pixel points in the three-by-three neighborhood; Taking the feature point as the center, extract an image block of thirty-two pixels by thirty-two pixels, divide the image block into four by four sub-blocks, calculate the grayscale mean of each sub-block to obtain a sixteen-dimensional feature vector, compare each component in the sixteen-dimensional feature vector with other components to obtain binary bits, combine the binary bits to form a two-hundred-and-fifty-six-bit binary feature descriptor, and use the position information of the feature point and the binary feature descriptor as feature data of the real-time image.

[0061] In an optional specific implementation, first, the acquired real-time image is decomposed at multiple scales to construct a pyramid structure including three layers of images. The original image resolution is used as the first layer of the pyramid. The original image resolution is reduced to half of the original image resolution to obtain the second layer of the pyramid image. The original image resolution is reduced to one quarter of the original image resolution to obtain the third layer of the pyramid image. For example, if the original image resolution is 1280x720 pixels, the second layer image resolution is 640x360 pixels, and the third layer image resolution is 320x180 pixels.

[0062] Then, feature point detection is performed on each layer of the constructed three-layer image pyramid. For each layer of the image, traverse each pixel in it. With the current pixel as the center, sixteen sampling points located on a virtual circle are uniformly selected around it. The grayscale difference between the central pixel and the sixteen sampling points is calculated. Assume that the grayscale value of the central pixel is 100, and the grayscale values ​​of the sixteen sampling points are 102, 105, 108, 110, 112, 115, 118, 120, 122, 125, 128, 130, 100, 98, 95, 92 respectively. The preset grayscale threshold is 10. Then the grayscale difference between the central pixel and the first eight sampling points is 2, 5, 8, 10, 12, 15, 18, 20 respectively, and the grayscale difference between the central pixel and the last eight sampling points is 22, 25, 28, 30, 0, -2, -5, -8 respectively.

[0063] Next, determine whether there are nine or more consecutive sampling points whose grayscale difference is greater than the preset grayscale threshold of 10. In this example, among the first eight sampling points, starting from the fourth sampling point, the grayscale difference of five consecutive sampling points is greater than 10. Therefore, the pixel is marked as a candidate feature point. Calculate the absolute value of the grayscale difference between the candidate feature point and the sixteen sampling points, and the difference with the preset grayscale threshold of 10, and take the maximum value as the response value of the candidate feature point. In this example, the absolute values ​​of the grayscale difference are 2, 5, 8, 10, 12, 15, 18, 20, 22, 25, 28, 30, 0, 2, 5, 8. The differences with the threshold of 10 are -8, -5, -2, 0, 2, 5, 8, 10, 12, 15, 18, 20, -10, -8, -5, -2. The maximum value is 20.

[0064] Perform non-maximum suppression on all candidate feature points. With the candidate feature point as the center, obtain all pixels in its 3x3 neighborhood and calculate the response values ​​of these pixels. If the response value of a candidate feature point is greater than the response values ​​of all other pixels in its 3x3 neighborhood, the candidate feature point is determined as a feature point.

[0065] For each feature point, extract a 32x32 pixel image block with it as the center. Divide the image block into 4x4 pixel sub-blocks, a total of 16 sub-blocks. Calculate the grayscale mean of each sub-block to obtain a 16-dimensional feature vector. Assume that the grayscale means of these 16 sub-blocks are 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 115, 120, 125 respectively. Compare each component in the feature vector with other components. For example, compare the first component 50 with other components. If 50 is less than other components, the corresponding binary bit is 0, otherwise it is 1. Finally, a 256-bit binary feature descriptor is obtained. The location information of the feature point and the corresponding 256-bit binary feature descriptor are used as the feature data of the real-time image.

[0066] In this embodiment, through multi-scale decomposition and fast feature point detection algorithm, the speed of feature extraction can be significantly improved to meet the real-time requirements; the binary feature descriptor is highly robust to image transformations such as illumination changes and rotation, which improves the accuracy of feature matching; the calculation and matching process of the binary feature descriptor is simple and efficient, which reduces the overall computational complexity and is conducive to deployment on resource-constrained platforms.

[0067] In an optional implementation, matching the feature data with the geometric feature parameters of the standard component, and obtaining the retained matching points by a projective transformation feature matching method includes: Perform feature matching based on the feature data and the geometric feature parameters, calculate the Hamming distance between the binary feature descriptor and the geometric feature parameters, find out the binary feature descriptor with the minimum Hamming distance and the second smallest Hamming distance for each geometric feature parameter, calculate the ratio of the minimum Hamming distance to the second smallest Hamming distance, and when the ratio is less than a preset threshold, determine the corresponding feature point position information and geometric feature parameters as a matching point pair; Randomly selecting three pairs of matching points from the matching point pairs, obtaining the coordinate position of each pair of matching points in the three pairs of matching points, constructing a coordinate mapping relationship according to the coordinate positions, establishing a linear equation group through the x-direction component and the y-direction component in the coordinate mapping relationship, solving the linear equation group to obtain transformation coefficients, and forming a transformation matrix for coordinate transformation with the transformation coefficients; Applying the transformation matrix to the position information of feature points in other matching point pairs except the three matching point pairs to obtain transformed position information, and calculating the Euclidean distance between the transformed position information and the geometric feature parameters in the corresponding matching point pairs as a reprojection error; Mark the matching point pairs whose reprojection errors are less than the error threshold as inliers that meet the transformation model, count the number of inliers of the inliers, compare the number of inliers with the current optimal number of inliers, and when the number of inliers is greater than the current optimal number of inliers, use the number of inliers to update the current optimal number of inliers, and use the corresponding transformation matrix to update the current optimal transformation matrix; The process is repeated until the preset maximum number of iterations is reached, and the inner point corresponding to the current optimal transformation matrix updated last is used as the reserved matching point.

[0068] The Hamming distance specifically refers to the degree of difference between two binary feature descriptors. For each pair of binary feature descriptors (assuming that each descriptor consists of a fixed length of binary bits), the Hamming distance is the number of different binary bits at corresponding positions. For example, if there are two descriptors 1011001 and 1001010, the Hamming distance between the two descriptors is 3, because the two descriptors have different values ​​at three positions (the 3rd, 4th, and 6th bits); when calculating the match between the descriptor and the geometric feature parameters, the matching quality is evaluated by comparing the Hamming distance. The smaller the Hamming distance, the more similar the two descriptors are, and thus the matching point is more likely to be a correct match.

[0069] The linear equation group specifically refers to a mathematical formula containing multiple linear equations, which is usually used to describe the linear relationship between multiple variables; in feature matching, based on three pairs of randomly selected matching points, the coordinate mapping relationship is established through the coordinate positions of the three pairs of matching points. These coordinate mapping relationships can be represented by a set of linear equations; by establishing these linear equations, the transformation coefficients can be calculated based on the known coordinates of the matching points. These coefficients describe the coordinate transformation relationship from the real-time image coordinate system to the standard coordinate system.

[0070] The transformation coefficient specifically refers to a numerical value used to describe the transformation relationship between two coordinate systems. The transformation coefficient is obtained from the coordinate position of the matching point pair and is used to convert points in one coordinate system (for example, feature points in a real-time image) to another coordinate system (for example, the geometric feature parameter coordinate system of a standard component). The transformation coefficient is usually calculated by minimizing errors (such as reprojection errors); including translation, scaling factors, rotation angles, etc., and the transformation matrix can be constructed through these coefficients to perform geometric transformation on the feature points in the image so that the feature points are aligned with the corresponding points in the standard component coordinate system.

[0071] Firstly, the binary feature descriptors are initially matched with the geometric feature parameters, and then the random sampling consistency algorithm is used to screen reliable matching point pairs.

[0072] The feature matching process first calculates the similarity between the binary feature descriptor extracted from the real-time image and the geometric feature parameters of the standard parts. For each binary feature descriptor to be matched, the Hamming distance between it and the geometric feature parameters is calculated. The Hamming distance reflects the degree of difference between the feature descriptor and the standard feature. For example, for a feature point to be matched, the minimum Hamming distance between its 256-bit binary feature descriptor and the corresponding geometric feature parameters is 12, and the second minimum Hamming distance is 42. The ratio of the calculated minimum value to the second minimum value is 0.286, which is less than the preset threshold of 0.5, so the corresponding feature point position information and geometric feature parameters are determined as valid matching point pairs.

[0073] Perform random sampling consistency verification on the initial matching point pairs. Randomly select three pairs of matching points and establish a coordinate mapping relationship based on the two-dimensional coordinates of these points. For example, among the three pairs of matching points selected, the coordinates of the real-time image feature points are (120, 160), (280, 220), and (350, 180), and the corresponding standard position coordinates in the geometric feature parameters are (125, 158), (288, 223), and (356, 183). Use these coordinate values ​​to establish a linear equation group in the x-direction and y-direction, and solve the transformation matrix consisting of six transformation coefficients. This matrix is ​​used to transform the image coordinates of the feature points into the standard coordinate system.

[0074] Use the obtained transformation matrix to verify other matching point pairs. Apply the transformation matrix to the feature point position information in the remaining matching point pairs to obtain the transformed position. Calculate the Euclidean distance between the transformed position and the standard position in the corresponding geometric feature parameters as the reprojection error. When the reprojection error is less than the threshold of 2 pixels, mark the matching point pair as an inlier. For example, the coordinates of a feature point after transformation are (245, 178), and the corresponding standard position in the geometric feature parameters is (246, 179). The calculated error value is 1.4 pixels, which is less than the threshold, so the matching point pair is retained.

[0075] Count the number of inliers currently obtained and compare it with the number of optimal inliers recorded. The current number of inliers is 85, which is greater than the previously recorded 78. Then update the optimal number of inliers to 85 and save the corresponding transformation matrix. Repeat the random sampling and verification process, and set the maximum number of iterations to 1000. Each iteration may find a better transformation model, and finally select the transformation matrix with the most inliers support as the optimal result.

[0076] Finally, the inner points corresponding to the optimal transformation matrix are used as reliable feature matching results. These matching points have high geometric consistency and can accurately reflect the positional correspondence between the parts to be assembled and the standard parts. For example, among the 85 pairs of matching points finally retained, the mean reprojection error is 1.2 pixels and the maximum error is 1.8 pixels, which reflects a high matching accuracy. These matching results can be used to subsequently calculate the assembly position deviation and posture deviation, and guide the precise implementation of the assembly process.

[0077] In this embodiment, by calculating the Hamming distance between the binary feature descriptor and the geometric feature parameters, the matching quality of each feature point can be effectively evaluated to ensure the high accuracy of the matching point pairs; a random sampling consistency algorithm is used to screen out reliable matching point pairs, and by calculating and verifying the optimal transformation matrix, it is ensured that the matching points finally retained have high geometric consistency, thereby reducing the interference of false matching on the assembly process; the transformation matrix is ​​continuously optimized through an iterative process, and finally the transformation matrix with the most inliers and the smallest reprojection error is selected, thereby further improving the matching accuracy and reliability and enhancing the assembly accuracy; the selection of the optimal transformation matrix and its inliers ensures the accurate positional correspondence between the components and the standard components, ultimately making the calculation of the position and posture deviation in the assembly process more accurate, thereby ensuring the high-precision implementation of automatic assembly; through a large number of iterations and comparisons of the number of inliers, the stability and reliability of the finally selected matching results in multiple tests are ensured, thereby providing an accurate positioning basis for the subsequent assembly process.

[0078] In an optional implementation, the robot is controlled to perform component assembly according to the updated assembly path point coordinates, and the force sensor data and displacement sensor data during the assembly process are collected and input into the Kalman filter for data fusion, and the assembly state evaluation index is calculated, including the slope of the assembly force curve, the displacement change rate, and the force-displacement relationship curve. Control the robot to perform component assembly according to the coordinates of the assembly path points, and collect force data output by the force sensor and displacement data output by the displacement sensor on the end effector of the robot, wherein the force data includes force components and torque components in three directions, and the displacement data includes displacement components in three directions; Constructing a state vector including the force data, the displacement data, the force change rate and the displacement change rate, constructing an observation vector including the force data and the displacement data, calculating the deviation between the state vector and the predicted state vector at the previous moment to obtain a state error covariance matrix; calculating a theoretical covariance matrix and an actual covariance matrix based on the state error covariance matrix, calculating the ratio of the trace of the theoretical covariance matrix to the trace of the actual covariance matrix, and obtaining an adaptive factor; The product of the adaptive factor and the process noise covariance at the previous moment is used as the current process noise covariance, and the current measurement noise covariance is calculated based on the difference sequence of the state vector and the observation vector; the state vector, the observation vector, the process noise covariance and the measurement noise covariance are input into an adaptive Kalman filter, and a filtered state vector is obtained through a prediction step and an update step of the filter; Based on the force data in the filtered state vector, the ratio of the force difference to the time difference between adjacent sampling periods is calculated to obtain the slope of the assembly force curve. Based on the displacement data in the filtered state vector, the ratio of the displacement difference to the time difference between adjacent sampling periods is calculated to obtain the displacement change rate. Based on the ratio of the force difference to the displacement difference, a force-displacement relationship curve is obtained.

[0079] In an optional specific implementation, first, the robot is controlled to assemble parts according to the predetermined assembly path point coordinates. The assembly path point coordinates describe the spatial position and posture sequence of the robot end effector during the assembly process. For example, in a simple linear insertion process, the path points can be set to (0, 0, 0), (0, 0, 10), (0, 0, 20)..., indicating that the robot end effector moves linearly along the Z-axis direction for insertion. During the robot's assembly task, the data of the force sensor and displacement sensor on the robot end effector are synchronously collected. The data output by the force sensor includes force components in three directions (for example, X-direction force Fx, Y-direction force Fy, Z-direction force Fz) and torque components in three directions (for example, X-direction torque Mx, Y-direction torque My, Z-direction torque Mz). The data output by the displacement sensor includes displacement components in three directions (for example, X-direction displacement Dx, Y-direction displacement Dy, Z-direction displacement Dz). Assume that at a certain moment, the collected force data is Fx=10N, Fy=2N, Fz=50N, Mx=0.5Nm, My=0.2Nm, Mz=0.1Nm, and the displacement data is Dx=5mm, Dy=1mm, Dz=20mm.

[0080] Next, construct the state vector and observation vector of the Kalman filter. The state vector contains force data, displacement data, force change rate, and displacement change rate. The observation vector contains force data and displacement data. For example, the state vector can be set to [Fx, Fy, Fz, Mx, My, Mz, Dx, Dy, Dz, dFx / dt, dFy / dt, dFz / dt, dMx / dt, dMy / dt, dMz / dt, dDx / dt, dDy / dt, dDz / dt], and the observation vector can be set to [Fx, Fy, Fz, Mx, My, Mz, Dx, Dy, Dz]. Calculate the difference between the current state vector and the predicted state vector at the previous moment to obtain the state error covariance matrix.

[0081] Then, the theoretical covariance matrix and the actual covariance matrix are calculated based on the state error covariance matrix. The ratio of the trace of the theoretical covariance matrix to the trace of the actual covariance matrix is ​​calculated to obtain the adaptive factor. Assuming that the calculated trace of the theoretical covariance matrix is ​​10 and the trace of the actual covariance matrix is ​​5, the adaptive factor is 10 / 5=2.

[0082] Multiply the adaptive factor by the process noise covariance of the previous moment to obtain the current process noise covariance. Assuming that the process noise covariance of the previous moment is 0.1, the current process noise covariance is 2×0.1=0.2. The current measurement noise covariance is calculated based on the difference sequence of the state vector and the observation vector. Assume that the calculated measurement noise covariance is 0.05.

[0083] The state vector, observation vector, process noise covariance and measurement noise covariance are input into the adaptive Kalman filter, and after the prediction step and update step of the filter, the filtered state vector is obtained. Assume that Fx=10.2N and Dx=5.1mm in the filtered state vector.

[0084] Finally, the assembly state evaluation index is calculated based on the filtered state vector. For example, the ratio of the force difference to the time difference between adjacent sampling cycles is calculated to obtain the slope of the assembly force curve. Assuming that Fx=10.2N at the current moment, Fx=10N at the previous moment, and the sampling period is 0.01 seconds, the slope of the assembly force curve is (10.2-10) / 0.01=20N / s. Calculate the ratio of the displacement difference to the time difference between adjacent sampling cycles to obtain the displacement change rate. Assuming that Dx=5.1mm at the current moment, Dx=5mm at the previous moment, and the sampling period is 0.01 seconds, the displacement change rate is (5.1-5) / 0.01=10mm / s. Calculate the ratio of the force difference to the displacement difference to obtain the force-displacement relationship curve. For example, (10.2-10) / (5.1-5)=20N / mm.

[0085] In this embodiment, by fusing the force sensor and displacement sensor data through the Kalman filter, the influence of sensor noise can be effectively reduced, and more accurate assembly status information can be obtained, thereby improving assembly accuracy; the assembly status evaluation index can be calculated in real time to provide real-time feedback for the robot control system so that the control strategy can be adjusted in time to ensure the stability and reliability of the assembly process; the adaptive Kalman filter can automatically adjust the filtering parameters according to the actual situation to adapt to different assembly environments and tasks, thereby improving the robustness and generalization ability of the method.

[0086] In an optional implementation, when the assembly state evaluation index exceeds a preset threshold, dynamically adjusting the assembly speed and assembly force of the robot according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed includes: Inputting the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve into a fuzzy neural network, converting the input variables into fuzzy variables through the fuzzification layer of the fuzzy neural network, performing fuzzy reasoning at the rule layer according to a preset fuzzy rule library, and calculating the first force adjustment amount and the first speed adjustment amount at the defuzzification layer; The slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are used as the state space of reinforcement learning, and the force adjustment amount and the speed adjustment amount are used as the action space; the deviation of the slope of the assembly force curve from a first preset threshold, the deviation of the displacement change rate from a second preset threshold, and the deviation of the force-displacement relationship curve from a third preset threshold are calculated, and a reward value is calculated based on multiple deviations; the state space, the action space and the reward value are input into a deep Q network, and a second force adjustment amount and a second speed adjustment amount are obtained through iterative updating of the Q value; Calculating the fuzzy control weight and the reinforcement learning weight according to the assembly state evaluation index in the historical assembly cycle; adding the product of the first force adjustment amount and the fuzzy control weight to the product of the second force adjustment amount and the reinforcement learning weight to obtain the final force adjustment amount, and adding the product of the first speed adjustment amount and the fuzzy control weight to the product of the second speed adjustment amount and the reinforcement learning weight to obtain the final speed adjustment amount; The final force adjustment amount is added to the current assembly force to obtain an updated assembly force, and the final speed adjustment amount is added to the current assembly speed to obtain an updated assembly speed; until the updated slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are all within the corresponding preset threshold ranges, the automatic assembly of parts is completed.

[0087] In an optional specific implementation, first, the force sensor data and displacement sensor data during the assembly process are collected. For example, in the scenario of assembling a bearing and a shaft, the force sensor and displacement sensor data can be collected every 0.1 seconds to record the force value and displacement of the bearing pressed into the shaft.

[0088] Then, based on the collected data, the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are calculated. The slope of the assembly force curve can be obtained by dividing the force difference between two adjacent time points by the time difference. The displacement change rate can be obtained by dividing the displacement difference between two adjacent time points by the time difference. The force-displacement relationship curve can be obtained by plotting the force value and displacement into a curve graph. For example, assuming that in a certain period of time, the force value increases from 10N to 20N, the displacement increases from 1mm to 2mm, and the time difference is 0.1 seconds, then the slope of the assembly force curve is (20-10) / 0.1=100N / s, and the displacement change rate is (2-1) / 0.1=10mm / s.

[0089] The calculated slope of the assembly force curve, displacement change rate and force-displacement relationship curve are input into the fuzzy neural network. The fuzzification layer of the fuzzy neural network converts these input variables into fuzzy variables, for example, converting the slope of the assembly force curve into fuzzy sets such as "large", "medium" and "small". Then, fuzzy reasoning is performed at the rule layer according to the preset fuzzy rule base. For example, a fuzzy rule may be: if the slope of the assembly force curve is "large" and the displacement change rate is "small", the force adjustment amount is "decrease" and the speed adjustment amount is "increase". Finally, the first force adjustment amount and the first speed adjustment amount are calculated in the defuzzification layer. For example, the first force adjustment amount is -0.5N and the first speed adjustment amount is 0.2mm / s.

[0090] At the same time, the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are used as the state space of reinforcement learning, and the force adjustment amount and the speed adjustment amount are used as the action space. Calculate the deviation of the slope of the assembly force curve from the first preset threshold, the deviation of the displacement change rate from the second preset threshold, and the deviation of the force-displacement relationship curve from the third preset threshold. For example, assuming that the first preset threshold is 80N / s, the deviation of the slope of the assembly force curve from the first preset threshold is 100-80=20N / s. Calculate the reward value based on these deviations. For example, the smaller the deviation, the greater the reward value. Input the state space, action space and reward value into the deep Q network, and obtain the second force adjustment amount and the second speed adjustment amount through Q value iterative update. For example, the second force adjustment amount is -0.2N, and the second speed adjustment amount is 0.1mm / s.

[0091] The fuzzy control weight and reinforcement learning weight are calculated based on the assembly state evaluation indicators (e.g., assembly accuracy, assembly time, etc.) in the historical assembly cycle. For example, if the historical data shows that the fuzzy control is more effective, a higher weight is given to the fuzzy control. The final force adjustment is obtained by adding the product of the first force adjustment and the fuzzy control weight to the product of the second force adjustment and the reinforcement learning weight. The final speed adjustment is obtained by adding the product of the first speed adjustment and the fuzzy control weight to the product of the second speed adjustment and the reinforcement learning weight. For example, assuming that the fuzzy control weight is 0.8 and the reinforcement learning weight is 0.2, the final force adjustment is -0.5×0.8+(-0.2)×0.2=-0.44N, and the final speed adjustment is 0.2×0.8+0.1×0.2=0.18mm / s.

[0092] The final force adjustment is added to the current assembly force to obtain the updated assembly force, and the final speed adjustment is added to the current assembly speed to obtain the updated assembly speed. For example, assuming that the current assembly force is 20N and the current assembly speed is 1mm / s, the updated assembly force is 20-0.44=19.56N, and the updated assembly speed is 1+0.18=1.18mm / s. Repeat the above steps until the slope of the updated assembly force curve, the displacement change rate, and the force-displacement relationship curve are all within the corresponding preset threshold ranges, and the automatic assembly of parts is completed.

[0093] In this embodiment, the assembly parameters can be dynamically adjusted according to the real-time data in the assembly process, which can effectively reduce the assembly error and improve the assembly accuracy. By optimizing the assembly speed and force, the assembly time can be shortened and the production efficiency can be improved. By integrating the advantages of fuzzy control and reinforcement learning, it can better adapt to the complex and changeable assembly environment and enhance the robustness of the assembly system.

[0094] Figure 2 FIG. 1 is a schematic diagram of the structure of an automatic assembly system for an automobile seat belt pretensioner based on machine vision according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to collect component images of the pretensioner standard component at different angles, perform grayscale processing and Gaussian filtering noise reduction on the component image, extract edge features of the component image using an edge detection operator, segment the target area of ​​the edge feature by a region growing method, calculate and extract geometric feature parameters of the target area, project the geometric feature parameters to a world coordinate system, establish a mapping relationship from two-dimensional image coordinates to three-dimensional space coordinates, and calculate the assembly reference point and assembly path point coordinates of the pretensioner standard component based on the mapping relationship; The second unit is used to collect real-time images of parts to be assembled, perform adaptive histogram equalization enhancement processing on the real-time images, extract feature data of the real-time images through a feature point detection algorithm; match the feature data with geometric feature parameters of standard parts, obtain reserved matching points through a projective transformation feature matching method, calculate the position deviation value and angle deviation value of the parts to be assembled relative to the assembly reference point based on the reserved matching points, calculate the compensation amount according to the position deviation value and the angle deviation value, and update the coordinates of the assembly path points; The third unit is used to control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation index, including the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve; when the assembly state evaluation index exceeds the preset threshold, the assembly speed and assembly force of the robot are dynamically adjusted according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed.

[0095] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0096] According to a fourth aspect of the embodiments of the present invention, A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0097] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The automatic assembly method of automobile seat belt pretensioner based on machine vision is characterized in that: include: Collect component images of pretensioner standard parts at different angles, perform grayscale processing and Gaussian filtering noise reduction on the component images, use edge detection operators to extract edge features of the component images, segment the target area of ​​the edge features by region growing method, calculate and extract geometric feature parameters of the target area, project the geometric feature parameters to the world coordinate system, establish a mapping relationship from two-dimensional image coordinates to three-dimensional space coordinates, and calculate the assembly reference point and assembly path point coordinates of the pretensioner standard parts based on the mapping relationship; Collecting real-time images of parts to be assembled, performing adaptive histogram equalization enhancement processing on the real-time images, extracting feature data of the real-time images through a feature point detection algorithm; matching the feature data with geometric feature parameters of standard parts, obtaining reserved matching points through a projective transformation feature matching method, calculating position deviation values ​​and angle deviation values ​​of the parts to be assembled relative to an assembly reference point based on the reserved matching points, calculating compensation amounts according to the position deviation values ​​and angle deviation values, and updating the coordinates of assembly path points; Control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation indicators, including the slope of the assembly force curve, the displacement change rate, and the force-displacement relationship curve; When the assembly state evaluation index exceeds a preset threshold, the assembly speed and assembly force of the robot are dynamically adjusted according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed.

2. The method according to claim 1, characterized in that Extracting edge features of the component image using an edge detection operator includes: Convolution operations are performed on the component image of the pretensioner standard component by using a first-order difference operator in the horizontal direction and a first-order difference operator in the vertical direction, respectively, to obtain a horizontal gradient value and a vertical gradient value of the component image; based on the horizontal gradient value and the vertical gradient value, a gradient amplitude and a gradient direction of each pixel point in the component image are calculated; Divide the component image into multiple local areas, calculate the gradient amplitude mean and the gradient amplitude standard deviation of each local area based on the gradient amplitude of each pixel point, calculate the local threshold of each local area according to the gradient amplitude mean and the gradient amplitude standard deviation, and use a bilinear interpolation method to smoothly transition the overlapping areas between the local areas; In each of the local areas, the gradient amplitude of each pixel is compared with the corresponding local threshold, and the gradient amplitude of the target pixel is compared with the gradient amplitude of the adjacent pixel points along the gradient direction corresponding to the target pixel point, and when the gradient amplitude of the target pixel point is greater than the local threshold and greater than the gradient amplitude of the adjacent pixel points, the target pixel point is marked as an edge point; Performing eight-neighborhood connectivity analysis on the edge point to obtain endpoint pixels, searching for connection points in a five-by-five neighborhood of the endpoint pixels, and connecting the two endpoint pixels when the distance between the two endpoint pixels is less than a preset distance threshold and the gradient direction difference is less than a preset angle threshold; The gradient amplitude of each edge point is interpolated along the gradient direction of each edge point, the edge point corresponding to the local maximum value is retained, the main edge point is determined, and the main edge point is processed by a morphological thinning algorithm to obtain an edge feature with a width of one pixel.

3. The method according to claim 2, characterized in that Segmenting the target region of the edge feature by using a region growing method, and calculating and extracting geometric feature parameters of the target region include: Calculate the local variance of the component image where the edge feature is located, select pixel points whose grayscale values ​​are greater than a first preset grayscale threshold and are not edge points in the component image as candidate seed points, calculate the grayscale mean and grayscale standard deviation of the pixel points in the eight neighborhoods of the candidate seed points, and when the grayscale standard deviation is less than a second preset grayscale threshold and the difference between the grayscale value of the pixel points in the eight neighborhoods and the grayscale mean is less than a third preset grayscale threshold, determine the candidate seed points whose mutual spacing is greater than a preset distance threshold as growth seed points, and add the growth seed points to a queue to be processed; Based on each growth seed point in the queue to be processed, each pixel point in the eight neighborhoods of the growth seed point is marked as a point to be grown, and the grayscale difference between each point to be grown and the corresponding growth seed point is calculated. When the grayscale difference is less than the initial threshold of grayscale similarity, the average grayscale difference between the point to be grown and the grown area is calculated. When the average grayscale difference is less than the initial threshold of regional consistency and the point to be grown does not cross the edge feature, the point to be grown is added to the queue to be processed and marked as a processed pixel point; During the growth process, each time a preset number of pixel points are processed, the grayscale standard deviation of the grown area is calculated, and based on the ratio of the grayscale standard deviation to the grayscale standard deviation of the eight neighborhoods corresponding to the growth seed point, the grayscale similarity initial threshold and the regional consistency initial threshold are updated respectively to obtain a new grayscale similarity threshold and a new regional consistency threshold, and the growth process is iteratively performed; Based on the sub-regions formed by the growth of multiple growth seed points, the grayscale difference between adjacent sub-regions and the gradient strength of the sub-region boundaries are calculated. When the grayscale difference is less than the difference threshold and the gradient strength is less than the gradient strength threshold, the adjacent sub-regions are merged to obtain a merged sub-region; the merged sub-region is optimized to remove the noise region with an area less than the area threshold, fill the holes in the region, and smooth the region boundary to obtain the target region corresponding to the edge feature; the center of gravity coordinates, area value, perimeter value and roundness value of the target region are calculated to obtain the geometric feature parameters of the target region.

4. The method according to claim 1, characterized in that: Extracting feature data of the real-time image by a feature point detection algorithm includes: Perform multi-scale decomposition on the real-time image to construct a three-layer image pyramid, wherein the first layer image uses the original resolution, the second layer image downsamples the original resolution to one-half, and the third layer image downsamples the original resolution to one-quarter; Perform feature point detection in each layer of the three-layer image pyramid, obtain sixteen sampling points on a circle centered on the pixel point for each pixel point in each layer of the image, and calculate the grayscale difference between the pixel point and the sixteen sampling points; When there are nine or more consecutive sampling points among the sixteen sampling points whose grayscale differences are greater than a preset grayscale threshold, the pixel point is marked as a candidate feature point, the absolute value of the grayscale difference between the candidate feature point and the sixteen sampling points is calculated, and the maximum value of the difference between the absolute value of the grayscale difference and the preset grayscale threshold is taken as the response value of the candidate feature point; Perform non-maximum suppression on the candidate feature point, obtain pixel points in a three-by-three neighborhood with the candidate feature point as the center, calculate the response values ​​of all pixel points in the three-by-three neighborhood, and determine the candidate feature point as a feature point when the response value of the candidate feature point is greater than the response values ​​of all pixel points in the three-by-three neighborhood; Taking the feature point as the center, extract an image block of thirty-two pixels by thirty-two pixels, divide the image block into four by four sub-blocks, calculate the grayscale mean of each sub-block to obtain a sixteen-dimensional feature vector, compare each component in the sixteen-dimensional feature vector with other components to obtain binary bits, combine the binary bits to form a two-hundred-and-fifty-six-bit binary feature descriptor, and use the position information of the feature point and the binary feature descriptor as feature data of the real-time image.

5. The method according to claim 4, characterized in that The feature data is matched with the geometric feature parameters of the standard parts, and the retained matching points obtained by the projection transformation feature matching method include: Perform feature matching based on the feature data and the geometric feature parameters, calculate the Hamming distance between the binary feature descriptor and the geometric feature parameters, find out the binary feature descriptor with the minimum Hamming distance and the second smallest Hamming distance for each geometric feature parameter, calculate the ratio of the minimum Hamming distance to the second smallest Hamming distance, and when the ratio is less than a preset threshold, determine the corresponding feature point position information and geometric feature parameters as a matching point pair; Randomly selecting three pairs of matching points from the matching point pairs, obtaining the coordinate position of each pair of matching points in the three pairs of matching points, constructing a coordinate mapping relationship according to the coordinate positions, establishing a linear equation group through the x-direction component and the y-direction component in the coordinate mapping relationship, solving the linear equation group to obtain transformation coefficients, and forming a transformation matrix for coordinate transformation with the transformation coefficients; Applying the transformation matrix to the position information of feature points in other matching point pairs except the three matching point pairs to obtain transformed position information, and calculating the Euclidean distance between the transformed position information and the geometric feature parameters in the corresponding matching point pairs as a reprojection error; Mark the matching point pairs whose reprojection errors are less than the error threshold as inliers that meet the transformation model, count the number of inliers of the inliers, compare the number of inliers with the current optimal number of inliers, and when the number of inliers is greater than the current optimal number of inliers, use the number of inliers to update the current optimal number of inliers, and use the corresponding transformation matrix to update the current optimal transformation matrix; The process is repeated until the preset maximum number of iterations is reached, and the inner point corresponding to the current optimal transformation matrix updated last is used as the reserved matching point.

6. The method according to claim 1, characterized in that Control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation indicators, including the slope of the assembly force curve, the displacement change rate, and the force-displacement relationship curve. Control the robot to perform component assembly according to the coordinates of the assembly path points, and collect force data output by the force sensor and displacement data output by the displacement sensor on the robot end effector, wherein the force data includes force components and torque components in three directions, and the displacement data includes displacement components in three directions; Constructing a state vector including the force data, the displacement data, the force change rate and the displacement change rate, constructing an observation vector including the force data and the displacement data, calculating the deviation between the state vector and the predicted state vector at the previous moment to obtain a state error covariance matrix; calculating a theoretical covariance matrix and an actual covariance matrix based on the state error covariance matrix, calculating the ratio of the trace of the theoretical covariance matrix to the trace of the actual covariance matrix, and obtaining an adaptive factor; The product of the adaptive factor and the process noise covariance at the previous moment is used as the current process noise covariance, and the current measurement noise covariance is calculated based on the difference sequence of the state vector and the observation vector; the state vector, the observation vector, the process noise covariance and the measurement noise covariance are input into an adaptive Kalman filter, and a filtered state vector is obtained through a prediction step and an update step of the filter; Based on the force data in the filtered state vector, the ratio of the force difference to the time difference between adjacent sampling periods is calculated to obtain the slope of the assembly force curve. Based on the displacement data in the filtered state vector, the ratio of the displacement difference to the time difference between adjacent sampling periods is calculated to obtain the displacement change rate. Based on the ratio of the force difference to the displacement difference, a force-displacement relationship curve is obtained.

7. The method according to claim 6, characterized in that When the assembly state evaluation index exceeds a preset threshold, dynamically adjusting the assembly speed and assembly force of the robot according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed includes: Inputting the slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve into a fuzzy neural network, converting the input variables into fuzzy variables through the fuzzification layer of the fuzzy neural network, performing fuzzy reasoning at the rule layer according to a preset fuzzy rule library, and calculating the first force adjustment amount and the first speed adjustment amount at the defuzzification layer; The slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are used as the state space of reinforcement learning, and the force adjustment amount and the speed adjustment amount are used as the action space; the deviation of the slope of the assembly force curve from a first preset threshold, the deviation of the displacement change rate from a second preset threshold, and the deviation of the force-displacement relationship curve from a third preset threshold are calculated, and a reward value is calculated based on multiple deviations; the state space, the action space and the reward value are input into a deep Q network, and a second force adjustment amount and a second speed adjustment amount are obtained through iterative updating of the Q value; Calculating the fuzzy control weight and the reinforcement learning weight according to the assembly state evaluation index in the historical assembly cycle; adding the product of the first force adjustment amount and the fuzzy control weight to the product of the second force adjustment amount and the reinforcement learning weight to obtain the final force adjustment amount, and adding the product of the first speed adjustment amount and the fuzzy control weight to the product of the second speed adjustment amount and the reinforcement learning weight to obtain the final speed adjustment amount; The final force adjustment amount is added to the current assembly force to obtain an updated assembly force, and the final speed adjustment amount is added to the current assembly speed to obtain an updated assembly speed; until the updated slope of the assembly force curve, the displacement change rate and the force-displacement relationship curve are all within the corresponding preset threshold ranges, the automatic assembly of parts is completed.

8. An automatic assembly system for automobile seat belt pretensioners based on machine vision, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect component images of the pretensioner standard component at different angles, perform grayscale processing and Gaussian filtering noise reduction on the component image, extract edge features of the component image using an edge detection operator, segment the target area of ​​the edge feature by a region growing method, calculate and extract geometric feature parameters of the target area, project the geometric feature parameters to a world coordinate system, establish a mapping relationship from two-dimensional image coordinates to three-dimensional space coordinates, and calculate the assembly reference point and assembly path point coordinates of the pretensioner standard component based on the mapping relationship; The second unit is used to collect real-time images of parts to be assembled, perform adaptive histogram equalization enhancement processing on the real-time images, extract feature data of the real-time images through a feature point detection algorithm; match the feature data with geometric feature parameters of standard parts, obtain reserved matching points through a projective transformation feature matching method, calculate the position deviation value and angle deviation value of the parts to be assembled relative to the assembly reference point based on the reserved matching points, calculate the compensation amount according to the position deviation value and the angle deviation value, and update the coordinates of the assembly path points; The third unit is used to control the robot to perform component assembly according to the updated assembly path point coordinates, collect force sensor data and displacement sensor data during the assembly process, input the data into the Kalman filter for data fusion, and calculate the assembly state evaluation indicators, including the slope of the assembly force curve, the displacement change rate, and the force-displacement relationship curve; When the assembly state evaluation index exceeds a preset threshold, the assembly speed and assembly force of the robot are dynamically adjusted according to the change trend of the force sensor data and the displacement sensor data until the automatic assembly is completed.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images