Robot motion control system and control method based on machine vision

Abstract

The invention discloses a robot motion control system and control method based on machine vision, relates to the technical field of machine vision, and provides effective data input for an error calculation and correction module by extracting a local feature point set Fp during operation of the system. And the error calculation and correction module calculates an image translation error Et and a rotation error Er based on the feature point set Fp, obtains a translation vector T and a rotation error R by correcting the image position, generates a complete panoramic image P, and provides reliable visual information support for subsequent target positioning and path planning. The motion control and target positioning module is combined with a panoramic image P and a positioning algorithm, so that the target positioning precision is improved, the path planning is optimized, the problems of image dislocation, positioning errors and path instability in a traditional method are solved, and the adaptive capacity and task execution efficiency of the robot in a dynamic environment are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a robot motion control system and a control method based on machine vision. Background Art

[0002] With the rapid development of modern technology, machine vision, as a core technology that enables machines to "see" and understand the world around them, has been widely used in all walks of life. Machine vision systems usually obtain image or video data through hardware devices such as cameras and sensors, and extract useful information through image processing algorithms to help automated equipment make decisions. In many fields such as automation, intelligent manufacturing, and medical testing, machine vision systems are used to achieve key tasks such as object recognition, quality inspection, and navigation control. Especially in robot control systems, machine vision technology can provide robots with "eyes" and "wisdom", enabling them to perceive the environment more accurately and perform corresponding actions.

[0003] In robot motion control, the accuracy of visual perception directly affects the robot's positioning, path planning and task execution capabilities. Especially in complex environments, robots need to constantly adjust their motion trajectories to avoid obstacles or accurately reach the target position. To achieve this, machine vision technology is often closely integrated with motion control systems to provide position feedback and correction strategies. For example, in multi-view image stitching and scene reconstruction, the robot's visual system needs to accurately align images from different perspectives to obtain a complete and accurate environmental model. Only through precise alignment can the robot achieve efficient and accurate motion control and avoid motion errors and positioning deviations.

[0004] However, although machine vision technology has been widely used in robot motion control systems, existing image alignment and correction methods still have certain limitations. Traditional image alignment methods usually rely on simple feature point matching or template-based comparative analysis, but in multi-view image stitching or complex scene reconstruction, misalignment between images still often occurs. This misalignment may come from slight changes in camera position, different shooting angles, or because of noise and distortion in the image itself, making it impossible to accurately align the image. Furthermore, although some algorithms can solve these problems to a certain extent, due to the lack of effective analysis of local features of the image, they often cannot handle image alignment tasks in complex backgrounds, especially in environments with multiple objects and dynamic changes.

[0005] Specifically, existing robot vision systems often face the following problems: First, the accuracy of image alignment is insufficient, resulting in obvious misalignment or distortion in the spliced ​​image; second, the ability to extract and analyze local image features (such as edges) is insufficient, and it cannot effectively cope with the challenges brought by different environments, lighting or perspective changes; third, most traditional algorithms cannot achieve automatic adjustment and correction, and require manual intervention or pre-setting, which greatly reduces the flexibility and automation level of the system. Therefore, in practical applications, the image alignment and correction process is still a bottleneck affecting the accuracy and efficiency of robot vision systems. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a robot motion control system and a control method based on machine vision, which solve the problems mentioned in the background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a robot motion control system based on machine vision, including an image acquisition and edge detection module, a feature extraction module, an error calculation and correction module, an image alignment and splicing module, and a motion control and target positioning module;

[0008] The image acquisition and edge detection module acquires image data of different perspectives based on the camera integrated in the robot, and uses the Canny edge detection algorithm to extract the edge of the image data, obtains the basic information of edge features in the image data, and forms an edge feature image Em;

[0009] The feature extraction module processes the edge feature basic information in the edge feature image Em to extract local feature points to form a feature point set Fp;

[0010] The error calculation and correction module calculates the image alignment error according to the acquired feature point set Fp, and corrects the position of the image according to the error to obtain the corrected translation vector T and image rotation error R;

[0011] The image alignment and stitching module aligns and stitches the images according to the acquired translation vector T and image rotation error R to generate a complete panoramic image P;

[0012] The motion control and target positioning module adjusts the target positioning and path planning during the robot motion by processing the panoramic image P and combining the positioning algorithm.

[0013] Preferably, the image acquisition and edge detection module includes an image acquisition unit and an image edge detection unit;

[0014] The image acquisition unit performs fixed visual inspection cycle shooting based on the camera integrated in the robot, and collects image data of different perspectives through the camera shooting, which is marked as raw image data Iraw;

[0015] The image edge detection unit grayscales the acquired original image data Iraw to obtain a grayscale image, and then uses the Canny edge detection algorithm to extract the edge of the image data, obtains the basic information of edge features in the image data, and forms an edge feature image Em, including the coordinate position (x, y) of the edge pixel, the edge intensity G(x, y) and the edge direction F(x, y).

[0016] Preferably, the edge pixel coordinate position (x, y) represents the two-dimensional coordinate position of the edge pixel in the image, and x and y represent the horizontal axis coordinate and the vertical axis coordinate in the two-dimensional coordinate respectively, which is specifically determined and obtained by a double threshold processing method;

[0017] The edge strength G(x, y) represents the gradient amplitude of edge pixels, which is used to distinguish strong edges from weak edges. The specific calculation formula is Get, where Gx and Gy represent the horizontal axis coordinate gradient value and the vertical axis coordinate gradient value respectively;

[0018] The edge direction F(x, y) represents the gradient direction of the edge pixel, which is used to describe the local directionality of the edge and distinguish horizontal, vertical and inclined edges. It is specifically obtained by the gradient method calculation formula F=atan2(Gy, Gx), where atan represents the inverse tangent function.

[0019] Preferably, the feature extraction module includes a feature extraction unit;

[0020] The feature extraction unit extracts the edge strength G(x, y) and edge direction F(x, y) from the edge feature basic information in the edge feature image Em, and then performs integration calculation to obtain the edge significance score S(x, y), which is used to judge the importance of the position coordinates (x, y) of the edge pixel in the local area, and the local significant points retained by non-maximum suppression form a significance score matrix Sz, and simultaneously selects feature points that meet the specific significance threshold Tthe from the significance score matrix Sz to obtain the feature point set Fp.

[0021] Preferably, the edge significance score S(x, y) is obtained by the following calculation formula:

[0022] S(x,y)=G(x,y)*|cos(F(x,y));

[0023] Where cos represents the cosine function;

[0024] The significance score matrix Sz is obtained by the following calculation formula:

[0025]

[0026] Where N(x, y) represents the local window centered at the position coordinate (x, y), max(S(u, v)) represents the upper limit of the edge significance score of the pixel point in the local window N(x, y), specifically the edge significance score at the position (u, v), otherwise represents that non-feature points are marked as 0;

[0027] The feature point set Fp is selected from the saliency score matrix Sz and the feature points that meet the saliency specific threshold Tthe are substituted into the feature point set Fp, and the selection is performed specifically through the following screening formula:

[0028] Fp={(xi,yi)Sz(xi,yi)>Tthe};

[0029] Where (xi, yi) represents the coordinate position (x, y) of the i-th feature point, Sz(xi, yi) represents the edge significance score S(x, y) of the i-th feature point at the coordinate position (x, y) in the significance score matrix Sz after non-maximum suppression, and Tthe represents the significance specific threshold.

[0030] Preferably, the error calculation and correction module includes an error calculation unit and an error correction unit;

[0031] The error calculation unit calculates the alignment error between the coordinate position (xi, yi) of the i-th feature point in the current image and the coordinate position (xj, yj) of the j-th feature point in the target image based on the acquired feature point set Fp, specifically selects the coordinate position (xi, yi) of the i-th feature point matched in the target image and the coordinate position (xj, yj) of the j-th feature point based on the feature point set Fp to calculate the position difference of each pair of feature points, obtains the translation error Et and the rotation error Er, and performs statistics on the overall error E between the current image and the target image;

[0032] The translation error Et is obtained by the following calculation formula:

[0033]

[0034] Wherein, △x represents the horizontal axis position difference between the coordinate positions of the i-th feature point and the j-th feature point on the horizontal axis coordinate, and △y represents the vertical axis position difference between the coordinate positions of the i-th feature point and the j-th feature point on the vertical axis coordinate;

[0035] The horizontal axis position difference △x is obtained by the calculation formula △x=xj-xi, where xj and xi represent the horizontal axis coordinates of the jth feature point and the ith feature point respectively;

[0036] The vertical axis position difference △y is obtained by the calculation formula △y=yj-yi, where yj and yi represent the vertical axis coordinates of the jth feature point and the ith feature point respectively;

[0037] The feature rotation error Er is obtained by the calculation formula Er=|F(xj,yj)-F(xi,yi), where F(xj,yj) represents the edge direction of the coordinate position of the jth feature point, and F(xi,yi) represents the edge direction of the coordinate position of the i-th feature point;

[0038] The overall error E is given by The calculation formula is obtained, where M represents the total number of matching feature points, m represents the mth pair of matching feature points, Et(m) and Er(m) represent the translation error Et and rotation error Er of the mth pair of matching feature points, respectively.

[0039] Preferably, the error correction unit obtains a translation vector T and an image rotation error R between the current image and the target image according to the translation error Et and the characteristic rotation error Er, so as to correct the deviation between the current image and the target image in the horizontal axis direction, the vertical axis direction and the rotation direction;

[0040] The translation vector T is obtained by the following calculation formula:

[0041]

[0042] Where M represents the total number of matching feature points, m represents the mth pair of matching feature points, and Et(m) represents the translation error Et of the mth pair of matching feature points.

[0043] The image rotation error R is Obtain the calculation formula.

[0044] Preferably, the image alignment and stitching module comprises aligning the images according to the acquired translation vector T and the image rotation error R, and then performing edge stitching on the aligned images to generate a complete panoramic image P;

[0045] Among them, the alignment is achieved by performing a translation transformation on each pixel in the image. Specifically, the pixel coordinate position (x, y) of the feature point is corrected by the translation vector T, and the corrected translation pixel coordinate position (x`, y`) is obtained and applied to the image for translation change, and the image after the translation change is rotated and corrected simultaneously. Specifically, the translation pixel coordinate position (x`, y`) is corrected in combination with the image rotation error R to obtain the corrected rotation pixel coordinate position (x``, y``);

[0046] The translation pixel coordinate position (x', y') is obtained by the calculation formula (x', y') = (x+T, y+T);

[0047] The rotated pixel coordinate position (x``, y``) is obtained by (x``, y``) = (x`*cos(R)-y`*sin(R), x`*sin(R)-y`*cos(R)), where cos(R) and sin(R) represent the cosine value and sine value of the image rotation error R, respectively.

[0048] Preferably, the motion control and target positioning module includes a control unit;

[0049] The control unit processes the panoramic image P and combines the positioning algorithm, including the target detection algorithm and the image segmentation algorithm, to extract the features of the target object in the panoramic image P, including corner points, local feature points and target shape, and obtains the relative position of the target object relative to the robot by matching the target features with the known target model in the robot environment and the target examples stored in the database, and adjusts the target positioning and path planning during the robot movement according to the obtained relative position, including the robot performing obstacle detection and path optimization according to the obtained relative position to replace the path motion trajectory generated by the robot in the previous visual detection cycle, and performing real-time adaptive dynamic adjustment.

[0050] A robot motion control method based on machine vision comprises the following steps:

[0051] S1. Collect image data from different perspectives based on the camera integrated with the robot, and use the Canny edge detection algorithm to extract the edge of the image data, obtain the basic information of edge features in the image data, and form an edge feature image Em;

[0052] S2, extracting local feature points by processing the edge feature basic information in the edge feature image Em to form a feature point set Fp;

[0053] S3, calculating the image alignment error based on the acquired feature point set Fp, and correcting the position of the image based on the error to obtain the corrected translation vector T and image rotation error R;

[0054] S4, aligning and stitching the images according to the obtained translation vector T and image rotation error R to generate a complete panoramic image P;

[0055] S5. By processing the panoramic image P and combining it with the positioning algorithm, the target positioning and path planning during the robot movement are adjusted.

[0056] The present invention provides a robot motion control system and control method based on machine vision, which has the following beneficial effects:

[0057] (1) When the system is running, by extracting the local feature point set Fp, it provides effective data input for the error calculation and correction module. The error calculation and correction module calculates the image translation error Et and rotation error Er based on the feature point set Fp, and obtains the translation vector T and rotation error R by correcting the image position, generating a complete panoramic image P, which provides reliable visual information support for subsequent target positioning and path planning. The motion control and target positioning module combines the panoramic image P and the positioning algorithm, which not only improves the target positioning accuracy, but also optimizes the path planning, enabling the robot to complete tasks accurately and efficiently in complex environments. It not only solves the problems of image misalignment, positioning error and path instability in traditional methods, but also greatly improves the robot's adaptability and task execution efficiency in dynamic environments, thereby achieving more intelligent and precise robot motion control.

[0058] (2) By calculating the translation error Et and rotation error Er of each pair of matching feature points, the difference between the current image and the target image is quantified, and these errors provide the necessary basis for subsequent image alignment. The translation error Et accurately reflects the spatial displacement between the two images by calculating the position difference of the feature points in the horizontal and vertical directions; the rotation error Er evaluates the rotation deviation of the two images by calculating the direction difference between the feature points. The translation vector T and rotation error R are obtained by statistically averaging, which effectively adjusts the position and angle of the image to ensure the accuracy of image alignment. This correction process not only reduces the misalignment problem in image stitching. By accurately correcting the image position and rotation, the robot can perform tasks more accurately, especially in complex visual navigation and target positioning, ensuring the efficiency and reliability of path planning and operation.

[0059] (3) The images are accurately aligned and stitched through the translation vector T and the rotation error R. The translation transformation ensures the correction of the coordinate position of the feature points in the image on the horizontal and vertical axes, while the rotation correction further optimizes the alignment effect of the image, thereby generating a complete and seamless panoramic image P. This process effectively eliminates the misalignment problem in image stitching and improves the accuracy and stitching effect of the robot vision system. Next, combined with the positioning algorithm, obstacle detection and path optimization are performed to achieve adaptive adjustment of path planning. In particular, for unknown obstacles in dynamic environments, the system can replace the previously generated motion trajectory in real time, ensuring the efficient movement and positioning of the robot in complex environments, greatly improving the robot's intelligent adaptability and path planning accuracy in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic diagram of a block diagram of a robot motion control system based on machine vision of the present invention;

[0061] Figure 2 The present invention is a schematic diagram of the steps of a robot motion control method based on machine vision. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0063] Example 1

[0064] The present invention provides a robot motion control system based on machine vision, please refer to Figure 1 , including image acquisition and edge detection module, feature extraction module, error calculation and correction module, image alignment and stitching module and motion control and target positioning module;

[0065] The image acquisition and edge detection module acquires image data of different perspectives based on the camera integrated in the robot, and uses the Canny edge detection algorithm to extract the edge of the image data, obtains the basic information of edge features in the image data, and forms an edge feature image Em;

[0066] The feature extraction module processes the edge feature basic information in the edge feature image Em to extract local feature points to form a feature point set Fp;

[0067] The error calculation and correction module calculates the image alignment error according to the acquired feature point set Fp, and corrects the position of the image according to the error to obtain the corrected translation vector T and image rotation error R;

[0068] The image alignment and stitching module aligns and stitches the images according to the acquired translation vector T and image rotation error R to generate a complete panoramic image P;

[0069] The motion control and target positioning module adjusts the target positioning and path planning during the robot motion by processing the panoramic image P and combining the positioning algorithm.

[0070] In this embodiment, by extracting edge feature information in the image, an edge feature image Em is constructed, which provides a basis for subsequent feature point extraction and matching. The feature extraction module successfully extracts the local feature point set Fp by processing the edge information in the edge feature image Em, and provides effective data input for the error calculation and correction module. The error calculation and correction module calculates the image translation error Et and rotation error Er based on the feature point set Fp, and obtains the translation vector T and rotation error R by correcting the image position, ensuring high-precision alignment of the image. The image alignment and splicing module uses the translation vector T and rotation error R to accurately splice the image to generate a complete panoramic image P, which provides reliable visual information support for subsequent target positioning and path planning. The motion control and target positioning module combines the panoramic image P and the positioning algorithm, which not only improves the target positioning accuracy, but also optimizes the path planning, so that the robot can accurately and efficiently complete the task in a complex environment. The collaboration between this series of modules not only solves the problems of image dislocation, positioning error and path instability in traditional methods, but also greatly improves the robot's adaptability and task execution efficiency in a dynamic environment, thereby realizing a more intelligent and precise robot motion control.

[0071] Example 2

[0072] This embodiment is explained in Example 1, please refer to Figure 1 , specifically: the image acquisition and edge detection module includes an image acquisition unit and an image edge detection unit;

[0073] The image acquisition unit performs fixed visual inspection cycle shooting based on the camera integrated in the robot, and collects image data of different perspectives through the camera shooting, which is marked as raw image data Iraw;

[0074] The image edge detection unit grayscales the acquired original image data Iraw to obtain a grayscale image, and then uses the Canny edge detection algorithm to extract the edge of the image data, obtains the basic information of edge features in the image data, and forms an edge feature image Em, including the coordinate position (x, y) of the edge pixel, the edge intensity G(x, y) and the edge direction F(x, y).

[0075] Wherein, the edge pixel coordinate position (x, y) represents the two-dimensional coordinate position of the edge pixel in the image, and x and y represent the horizontal axis coordinate and the vertical axis coordinate in the two-dimensional coordinate, respectively, which are specifically determined and obtained by the double threshold processing method;

[0076] The edge strength G(x, y) represents the gradient amplitude of edge pixels, which is used to distinguish strong edges from weak edges. The specific calculation formula is Get, where Gx and Gy represent the horizontal axis coordinate gradient value and the vertical axis coordinate gradient value respectively;

[0077] The edge direction F(x, y) represents the gradient direction of the edge pixel, which is used to describe the local directionality of the edge and distinguish horizontal, vertical and inclined edges. It is obtained by the gradient method calculation formula F=atan2(Gy, Gx), where atan represents the inverse tangent function, which is specifically used to calculate the angle of a given tangent value. atan2 represents an improved version of the inverse tangent function, which can receive two adopted numbers Gy and Gx and return a complete angle, taking into account four quadrants and avoiding quadrant problems that may occur in atan calculation.

[0078] In this embodiment, the robot integrated camera is used to capture images according to a fixed visual detection cycle and marked as raw image data Iraw, ensuring that the acquired image data has rich information from multiple angles and perspectives, laying the foundation for further analysis. Then, the image edge detection unit grayscales the raw image data Iraw to generate a grayscale image, and accurately extracts the edge features in the image through the Canny edge detection algorithm to construct an edge feature image Em. Among them, the edge pixel coordinate position (x, y) provides the precise edge position in the image; the edge intensity G (x, y) distinguishes the strong edge from the weak edge in the image, which helps to improve the accuracy of subsequent processing; the edge direction F (x, y) accurately describes the local directionality of each edge in the image, ensuring the effective identification and processing of edges with different directions. Through the extraction of these key feature information, the module provides more accurate visual data input for subsequent operations such as feature point extraction and image alignment, enhancing the overall image processing capability of the system, especially the adaptability and robustness in complex environments. This process not only ensures the high-quality acquisition of data, but also provides a solid foundation for subsequent target positioning and path planning.

[0079] Example 3

[0080] This embodiment is explained in Example 2. Please refer to Figure 1 ,Specifically: the feature extraction module includes a feature extraction unit;

[0081] The feature extraction unit extracts the edge strength G(x, y) and edge direction F(x, y) from the edge feature basic information in the edge feature image Em, and then performs integration calculation to obtain the edge significance score S(x, y), which is used to judge the importance of the position coordinates (x, y) of the edge pixel in the local area, and the local significant points retained by non-maximum suppression form a significance score matrix Sz, and simultaneously selects feature points that meet the specific significance threshold Tthe from the significance score matrix Sz to obtain the feature point set Fp.

[0082] The edge significance score S(x, y) is obtained by the following calculation formula:

[0083] S(x,y)=G(x,y)*|cos(F(x,y));

[0084] In the formula, cos represents the cosine function, and |cos(F(x, y))| is specifically used to quantify the relative relationship between the gradient direction and the horizontal and vertical coordinates. By calculating the cosine value of F(x, y), the directionality is eliminated and the feature strength is highlighted, so that the saliency in any direction of the acquired image can be evaluated.

[0085] The significance score matrix Sz is obtained by the following calculation formula:

[0086]

[0087] Wherein, N(x, y) represents the local window centered on the position coordinate (x, y), specifically a region of size k*k, and performs non-maximum suppression operation in region k*k to obtain feature points with the upper limit of the edge significance score in the local region k*k, max(S(u, v)) represents obtaining the upper limit of the edge significance score of the pixel point in the local window N(x, y), specifically the edge significance score at the position (u, v), otherwise represents that non-feature points are marked as 0;

[0088] The feature point set Fp is selected from the saliency score matrix Sz and the feature points that meet the saliency specific threshold Tthe are substituted into the feature point set Fp, and the selection is performed specifically through the following screening formula:

[0089] Fp={(xi,yi)Sz(xi,yi)>Tthe};

[0090] Wherein, (xi, yi) represents the coordinate position (x, y) of the i-th feature point, xi and yi specifically represent the horizontal and vertical coordinates of the i-th feature point, Sz(xi, yi) represents the edge significance score S(x, y) of the i-th feature point at the coordinate position (x, y) in the significance score matrix Sz after non-maximum suppression, and Tthe represents the significance specific threshold, and the specific value is set by the user.

[0091] In this embodiment, by extracting the edge strength G(x, y) and edge direction F(x, y) in the edge feature image Em, and calculating the edge significance score S(x, y), a quantitative significance value is assigned to each edge pixel point, which helps the system to accurately evaluate the edge importance of different regions, avoids the influence of directional deviation, effectively highlights the edge strength, and enables the significant features in the image to be balanced in all directions. Next, the significance score matrix Sz obtains the most significant edge feature points in the local area by applying non-maximum suppression in the local window N(x, y). These significant feature points are filtered out from the matrix through a screening formula to select a feature point set Fp that meets the threshold condition, providing high-quality input for subsequent feature matching, image alignment, and target positioning operations. The quality and quantity of feature points are optimized, ensuring that in complex scenes, the robot can perform positioning and path planning based on more accurate image features, effectively improving the accuracy and stability of the overall system.

[0092] Example 4

[0093] This embodiment is explained in Example 3, please refer to Figure 1 ,Specifically: the error calculation and correction module includes an error calculation unit and an error correction unit;

[0094] The error calculation unit calculates the alignment error between the coordinate position (xi, yi) of the i-th feature point in the current image and the coordinate position (xj, yj) of the j-th feature point in the target image based on the acquired feature point set Fp, specifically selects the coordinate position (xi, yi) of the i-th feature point matched in the target image and the coordinate position (xj, yj) of the j-th feature point based on the feature point set Fp to calculate the position difference of each pair of feature points, obtains the translation error Et and the rotation error Er, and performs statistics on the overall error E between the current image and the target image;

[0095] The translation error Et is obtained by the following calculation formula:

[0096]

[0097] Wherein, △x represents the horizontal axis position difference between the coordinate positions of the i-th feature point and the j-th feature point on the horizontal axis coordinate, and △y represents the vertical axis position difference between the coordinate positions of the i-th feature point and the j-th feature point on the vertical axis coordinate;

[0098] The horizontal axis position difference △x is obtained by the calculation formula △x=xj-xi, where xj and xi represent the horizontal axis coordinates of the jth feature point and the ith feature point respectively;

[0099] The vertical axis position difference △y is obtained by the calculation formula △y=yj-yi, where yj and yi represent the vertical axis coordinates of the jth feature point and the ith feature point respectively;

[0100] The feature rotation error Er is obtained by the calculation formula Er=|F(xj,yj)-F(xi,yi), where F(xj,yj) represents the edge direction of the coordinate position of the jth feature point, and F(xi,yi) represents the edge direction of the coordinate position of the i-th feature point;

[0101] The overall error E is given by The calculation formula is obtained, where M represents the total number of matching feature points, m represents the mth pair of matching feature points, Et(m) and Er(m) represent the translation error Et and rotation error Er of the mth pair of matching feature points, respectively.

[0102] The error correction unit obtains the translation vector T and the image rotation error R between the current image and the target image according to the translation error Et and the characteristic rotation error Er, so as to correct the deviation between the current image and the target image in the horizontal axis direction, the vertical axis direction and the rotation direction;

[0103] The translation vector T is obtained by the following calculation formula:

[0104]

[0105] Where M represents the total number of matching feature points, m represents the mth pair of matching feature points, and Et(m) represents the translation error Et of the mth pair of matching feature points.

[0106] The image rotation error R is Obtain the calculation formula.

[0107] In this embodiment, the difference between the current image and the target image is quantified by calculating the translation error Et and rotation error Er of each pair of matching feature points, and these errors provide the necessary basis for subsequent image alignment. The translation error Et accurately reflects the spatial displacement between the two images by calculating the position difference of the feature points in the horizontal and vertical directions; the rotation error Er evaluates the rotation deviation of the two images by calculating the direction difference between the feature points. The translation vector T and the rotation error R are obtained by statistically averaging, which effectively adjusts the position and angle of the image to ensure the accuracy of image alignment. This correction process not only reduces the misalignment problem in image stitching, but also improves the stability and adaptability of the robot positioning system in a dynamic environment. By accurately correcting the image position and rotation, the robot can perform tasks more accurately, especially in complex visual navigation and target positioning, ensuring the efficiency and reliability of path planning and operation.

[0108] Example 5

[0109] This embodiment is explained in Example 4. Please refer to Figure 1 Specifically: the image alignment and stitching module includes aligning the images according to the obtained translation vector T and the image rotation error R, and then stitching the edges of the aligned images to generate a complete panoramic image P;

[0110] Among them, the alignment is achieved by performing a translation transformation on each pixel in the image. Specifically, the pixel coordinate position (x, y) of the feature point is corrected by the translation vector T, and the corrected translation pixel coordinate position (x`, y`) is obtained and applied to the image for translation change, and the image after the translation change is rotated and corrected simultaneously. Specifically, the translation pixel coordinate position (x`, y`) is corrected in combination with the image rotation error R to obtain the corrected rotation pixel coordinate position (x``, y``);

[0111] The translation pixel coordinate position (x', y') is obtained by the calculation formula (x', y') = (x+T, y+T);

[0112] The rotated pixel coordinate position (x``, y``) is obtained by (x``, y``) = (x`*cos(R)-y`*sin(R), x`*sin(R)-y`*cos(R)), where cos(R) and sin(R) respectively represent the cosine value and sine value of the image rotation error R, which are specifically used to determine the coordinate transformation amplitude after rotation.

[0113] The motion control and target positioning module includes a control unit;

[0114] The control unit processes the panoramic image P and combines the positioning algorithm, including the target detection algorithm and the image segmentation algorithm, to extract the features of the target object in the panoramic image P, including corner points, local feature points and target shape, and obtains the relative position of the target object relative to the robot by matching the target features with the known target model in the robot environment and the target examples stored in the database, and adjusts the target positioning and path planning during the robot movement according to the obtained relative position, including the robot performing obstacle detection and path optimization according to the obtained relative position to replace the path motion trajectory generated by the robot in the previous visual detection cycle, and performing real-time adaptive dynamic adjustment.

[0115] In this embodiment, the images are accurately aligned and spliced ​​by the translation vector T and the rotation error R. The translation transformation ensures the correction of the coordinate position of the feature points in the image on the horizontal and vertical axes, and the rotation correction further optimizes the alignment effect of the image, thereby generating a complete and seamless panoramic image P. This process effectively eliminates the misalignment problem in image splicing and improves the accuracy and splicing effect of the robot vision system. Next, the motion control and target positioning module extracts the features of the target object from the panoramic image P, combines the positioning algorithm, including target detection and image segmentation algorithm, to accurately identify and match the target object with the known target model in the robot environment. By obtaining the relative position of the target in real time, the system can dynamically adjust the robot's motion path, perform obstacle detection and path optimization, thereby realizing adaptive adjustment of path planning. In particular, for unknown obstacles in a dynamic environment, the system can replace the previously generated motion trajectory in real time to ensure the efficient motion and positioning of the robot in a complex environment, greatly improving the robot's intelligent adaptability and path planning accuracy in practical applications.

[0116] Example 6

[0117] A robot motion control method based on machine vision, please refer to Figure 2 , specifically: including the following steps:

[0118] S1. Collect image data from different perspectives based on the camera integrated with the robot, and use the Canny edge detection algorithm to extract the edge of the image data, obtain the basic information of edge features in the image data, and form an edge feature image Em;

[0119] S2, extracting local feature points by processing the edge feature basic information in the edge feature image Em to form a feature point set Fp;

[0120] S3, calculating the image alignment error based on the acquired feature point set Fp, and correcting the position of the image based on the error to obtain the corrected translation vector T and image rotation error R;

[0121] S4, aligning and stitching the images according to the obtained translation vector T and image rotation error R to generate a complete panoramic image P;

[0122] S5. By processing the panoramic image P and combining it with the positioning algorithm, the target positioning and path planning during the robot movement are adjusted.

[0123] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A robot motion control system based on machine vision, characterized in that: It includes image acquisition and edge detection module, feature extraction module, error calculation and correction module, image alignment and splicing module and motion control and target positioning module; The image acquisition and edge detection module acquires image data of different perspectives based on the camera integrated in the robot, and uses the Canny edge detection algorithm to extract the edge of the image data, obtains the basic information of edge features in the image data, and forms an edge feature image Em; The feature extraction module processes the edge feature basic information in the edge feature image Em to extract local feature points to form a feature point set Fp; The error calculation and correction module calculates the image alignment error according to the acquired feature point set Fp, and corrects the position of the image according to the error to obtain the corrected translation vector T and image rotation error R; The image alignment and stitching module aligns and stitches the images according to the acquired translation vector T and image rotation error R to generate a complete panoramic image P; The motion control and target positioning module adjusts the target positioning and path planning during the robot motion by processing the panoramic image P and combining the positioning algorithm.

2. A robot motion control system based on machine vision according to claim 1, characterized in that: The image acquisition and edge detection module includes an image acquisition unit and an image edge detection unit; The image acquisition unit performs fixed visual inspection cycle shooting based on the camera integrated in the robot, and collects image data of different perspectives through the camera shooting, which is marked as raw image data Iraw; The image edge detection unit grayscales the acquired original image data Iraw to obtain a grayscale image, and then uses the Canny edge detection algorithm to extract the edge of the image data, obtains the basic information of edge features in the image data, and forms an edge feature image Em, including the coordinate position (x, y) of the edge pixel, the edge intensity G(x, y) and the edge direction F(x, y).

3. A robot motion control system based on machine vision according to claim 2, characterized in that: in, The edge pixel coordinate position (x, y) represents the two-dimensional coordinate position of the edge pixel in the image, where x and y represent the horizontal axis coordinate and the vertical axis coordinate in the two-dimensional coordinate, respectively, and is specifically obtained by judging through a double threshold processing method; The edge strength G(x, y) represents the gradient amplitude of edge pixels, which is used to distinguish strong edges from weak edges. The specific calculation formula is Get, where Gx and Gy represent the horizontal axis coordinate gradient value and the vertical axis coordinate gradient value respectively; The edge direction F(x, y) represents the gradient direction of the edge pixel, which is used to describe the local directionality of the edge and distinguish horizontal, vertical and inclined edges. It is specifically obtained by the gradient method calculation formula F=atan2(Gy, Gx), where atan represents the inverse tangent function.

4. A robot motion control system based on machine vision according to claim 3, characterized in that: The feature extraction module includes a feature extraction unit; The feature extraction unit extracts the edge strength G(x, y) and edge direction F(x, y) from the edge feature basic information in the edge feature image Em, and then performs integration calculation to obtain the edge significance score S(x, y), which is used to judge the importance of the position coordinates (x, y) of the edge pixel in the local area, and the local significant points retained by non-maximum suppression form a significance score matrix Sz, and simultaneously selects feature points that meet the specific significance threshold Tthe from the significance score matrix Sz to obtain the feature point set Fp.

5. The robot motion control system based on machine vision according to claim 4, characterized in that: The edge significance score S(x, y) is obtained by the following calculation formula: S(x,y)=G(x,y)* | cos(F(x,y)); In the formula, cos represents the cosine function; The significance score matrix Sz is obtained by the following calculation formula: Where N(x, y) represents the local window centered at the position coordinate (x, y), max(S(u, v)) represents the upper limit of the edge significance score of the pixel point in the local window N(x, y), specifically the edge significance score at the position (u, v), otherwise represents that non-feature points are marked as 0; The feature point set Fp is selected from the saliency score matrix Sz and the feature points that meet the saliency specific threshold Tthe are substituted into the feature point set Fp, and the selection is performed specifically through the following screening formula: Fp={(xi,yi)Sz(xi,yi)>Tthe}; Where (xi, yi) represents the coordinate position (x, y) of the i-th feature point, Sz(xi, yi) represents the edge significance score S(x, y) of the i-th feature point at the coordinate position (x, y) in the significance score matrix Sz after non-maximum suppression, and Tthe represents the significance specific threshold.

6. The robot motion control system based on machine vision according to claim 5, characterized in that: The error calculation and correction module includes an error calculation unit and an error correction unit; The error calculation unit calculates the alignment error between the coordinate position (xi, yi) of the i-th feature point in the current image and the coordinate position (xj, yj) of the j-th feature point in the target image based on the acquired feature point set Fp, specifically selects the coordinate position (xi, yi) of the i-th feature point matched in the target image and the coordinate position (xj, yj) of the j-th feature point based on the feature point set Fp to calculate the position difference of each pair of feature points, obtains the translation error Et and the rotation error Er, and performs statistics on the overall error E between the current image and the target image; The translation error Et is obtained by the following calculation formula: Wherein, △x represents the horizontal axis position difference between the coordinate positions of the i-th feature point and the j-th feature point on the horizontal axis coordinate, and △y represents the vertical axis position difference between the coordinate positions of the i-th feature point and the j-th feature point on the vertical axis coordinate; The horizontal axis position difference △x is obtained by the calculation formula △x=xj-xi, where xj and xi represent the horizontal axis coordinates of the jth feature point and the ith feature point respectively; The vertical axis position difference △y is obtained by the calculation formula △y=yj-yi, where yj and yi represent the vertical axis coordinates of the jth feature point and the ith feature point respectively; The characteristic rotation error Er is expressed by Er= | The calculation formula F(xj,yj)-F(xi,yi) is obtained, where F(xj,yj) represents the edge direction of the coordinate position of the jth feature point, and F(xi,yi) represents the edge direction of the coordinate position of the i-th feature point; The overall error E is given by The calculation formula is obtained, where M represents the total number of matching feature points, m represents the mth pair of matching feature points, Et(m) and Er(m) represent the translation error Et and rotation error Er of the mth pair of matching feature points, respectively.

7. The robot motion control system based on machine vision according to claim 6, characterized in that: The error correction unit obtains the translation vector T and the image rotation error R between the current image and the target image according to the translation error Et and the characteristic rotation error Er, so as to correct the deviation between the current image and the target image in the horizontal axis direction, the vertical axis direction and the rotation direction; The translation vector T is obtained by the following calculation formula: Where M represents the total number of matching feature points, m represents the mth pair of matching feature points, and Et(m) is the translation error Et of the mth pair of matching feature points. The image rotation error R is Obtain the calculation formula.

8. The robot motion control system based on machine vision according to claim 7, characterized in that: The image alignment and stitching module includes aligning the images according to the obtained translation vector T and the image rotation error R, and then stitching the edges of the aligned images to generate a complete panoramic image P; Among them, the alignment is achieved by performing a translation transformation on each pixel in the image. Specifically, the pixel coordinate position (x, y) of the feature point is corrected by the translation vector T, and the corrected translation pixel coordinate position (x`, y`) is obtained and applied to the image for translation change, and the image after the translation change is rotated and corrected simultaneously. Specifically, the translation pixel coordinate position (x`, y`) is corrected in combination with the image rotation error R to obtain the corrected rotation pixel coordinate position (x``, y``); The translation pixel coordinate position (x', y') is obtained by the calculation formula (x', y') = (x+T, y+T); The rotated pixel coordinate position (x``, y``) is obtained by (x``, y``) = (x`*cos(R)-y`*sin(R), x`*sin(R)-y`*cos(R)), where cos(R) and sin(R) represent the cosine value and sine value of the image rotation error R, respectively.

9. The robot motion control system based on machine vision according to claim 2, characterized in that: The motion control and target positioning module includes a control unit; The control unit processes the panoramic image P and combines the positioning algorithm, including the target detection algorithm and the image segmentation algorithm, to extract the features of the target object in the panoramic image P, including corner points, local feature points and target shape, and obtains the relative position of the target object relative to the robot by matching the target features with the known target model in the robot environment and the target examples stored in the database, and adjusts the target positioning and path planning during the robot movement according to the obtained relative position, including the robot performing obstacle detection and path optimization according to the obtained relative position to replace the path motion trajectory generated by the robot in the previous visual detection cycle, and performing real-time adaptive dynamic adjustment.

10. A robot motion control method based on machine vision, applied to a robot motion control system based on machine vision according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1. Collect image data from different perspectives based on the camera integrated with the robot, and use the Canny edge detection algorithm to extract the edge of the image data, obtain the basic information of edge features in the image data, and form an edge feature image Em; S2, extracting local feature points by processing the edge feature basic information in the edge feature image Em to form a feature point set Fp; S3, calculating the image alignment error based on the acquired feature point set Fp, and correcting the position of the image based on the error to obtain the corrected translation vector T and image rotation error R; S4, aligning and splicing the images according to the obtained translation vector T and image rotation error R to generate a complete panoramic image P; S5. By processing the panoramic image P and combining it with the positioning algorithm, the target positioning and path planning during the robot movement are adjusted.

Images