Robot space vision servo control method and system based on orthogonal vision system

Abstract

The invention belongs to the technical field related to robot control, and discloses a robot space vision servo control method and system based on an orthogonal vision system. The method comprises the following steps: respectively acquiring images of a first target object and a second target object to obtain a pixel A of the first target object and a pixel B of the second target object; the minimum pixel error between the pixel C1 of the reference object in the XOY plane and the pixel A of the first target object is taken as a target, and the robot is controlled to approach the first target object in the XOY plane until the distance between the first target object and the robot meets a preset first threshold value; and with the minimum pixel error between the real-time pixel C2 of the reference object and the pixel A of the first target object and the minimum pixel error between the real-time pixel D of the second target and the pixel B as the target, the robot is controlled to approach the first target object and the second target object in the XYZ plane until a preset second threshold value is met, and therefore control over the robot is achieved. According to the invention, high-precision spatial positioning of the target is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field related to robot control, and more specifically, relates to a robot space visual servo control method and system based on an orthogonal vision system. Background Art

[0002] With the continuous development of the manufacturing industry, the demand for high-precision robot processing is growing. While the number of processing tasks is increasing rapidly, the requirements for processing quality are becoming more stringent. Among the many robot control technologies, the visual servo system has become a key solution for achieving high-precision spatial positioning and operation due to its flexibility, high efficiency and robustness in complex environments.

[0003] Visual servo systems are mainly divided into position-based visual servo (PBVS) and image-based visual servo (IBVS). PBVS input is calculated in three-dimensional space, and the robot motion is controlled by estimating the position of image features in three-dimensional space. It is very sensitive to calibration errors and environmental changes. IBVS directly calculates control input in two-dimensional image space and controls robot motion by adjusting the position of the target in the image. Its main advantage is that it has high robustness to calibration errors, simple calculation, strong real-time performance, and is particularly suitable for tasks in dynamic and complex environments.

[0004] According to the installation method of the camera, the visual servo system is divided into two types: eye-in-hand and eye-to-hand. The eye-in-hand camera can dynamically adjust the field of view to observe the target object, which is suitable for achieving high spatial positioning accuracy, but it is difficult to directly observe the relative position between the end tool and the target, and requires high hand-eye calibration accuracy. The eye-to-hand camera is fixed in the workspace and can observe the target and the robot end at the same time. It is suitable for multi-robot collaboration and complex workpiece processing tasks, and can cover a larger workspace, but its accuracy is limited by the camera's resolution and observation distance. Therefore, a method is needed to simultaneously solve the above-mentioned eye-in-hand and eye-to-hand problems. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a robot spatial visual servo control method and system based on an orthogonal vision system, which solves the problems of high requirements for camera intrinsic parameter calibration and hand-eye calibration accuracy, poor real-time performance, and the accuracy in the XY direction is limited by the camera resolution, and the accuracy in the Z direction is limited by the observation distance of the camera.

[0006] To achieve the above object, according to one aspect of the present invention, a robot spatial visual servo control method based on an orthogonal vision system is provided, the method comprising the following steps:

[0007] Collecting images of the first target object and the second target object respectively, and performing image processing on the collected images to obtain pixels A of the first target object and pixels B of the second target object respectively;

[0008] The robot moves in the XOY plane, collects the image of the reference object in real time and obtains the pixel C1 of the reference object in the XOY plane, and controls the robot to approach the first target object in the XOY plane with the goal of minimizing the pixel error between the pixel C1 of the reference object in the XOY plane and the pixel A of the first target object until the distance between the two meets a preset first threshold;

[0009] The robot moves in the XYZ plane, collects the image of the reference object and the image of the second target object in real time and obtains the real-time pixels of the reference object and the second target in the image, which are the real-time pixel C2 of the reference object and the real-time pixel D of the second target, respectively. The pixel error between the real-time pixel C2 of the reference object and the pixel A of the first target, and the pixel error between the real-time pixel D and the pixel B of the second target are minimized, respectively. The robot is controlled to approach the first target object and the second target object in the XYZ plane until a preset second threshold is met, thereby realizing the control of the robot.

[0010] Further preferably, when the robot is controlled to approach the first target object in the XOY plane and when the robot is controlled to approach the first target object and the second target object in the XYZ plane, the movement speed of the robot is calculated according to the following formula:

[0011] v e =-Adj(T)L + λe

[0012] Among them, v e is the robot terminal velocity, T is the hand-eye matrix of the first image acquisition device or the second image acquisition device obtained by calibration, Adj(T) is the velocity adjoint matrix, L + is the Moore-Penrose pseudo-inverse of L, λ is a constant used to control the relative velocity of the robot, L is the visual servo matrix, and e is the pixel error.

[0013] Further preferably, the visual servo matrix L is as follows:

[0014]

[0015] Among them, (u, v) is the real-time coordinate of the pixel point in the image of the reference object or the second target object at the current moment, (u 0 ,v 0 ) is the optical center coordinate of the first image acquisition device or the second image acquisition device, f x ,f yis the focal length of the first image acquisition device or the second image acquisition device along the X direction and the Y direction, Z C It is the spatial depth of the image of the reference object or the second target object at the current moment.

[0016] Further preferably, the formula of the pixel error is as follows:

[0017] e(t)=ss *

[0018] Where s is the pixel C1 of the reference object in the XOY plane, the real-time pixel C2 of the reference object or the real-time pixel D of the second target object, s * is pixel A of the first object or pixel B of the second object, and t is the time.

[0019] Further preferably, the method for acquiring the pixel A of the first target object is as follows:

[0020] Performing smoothing and noise reduction processing on the collected image of the first target object and extracting edge features in the image;

[0021] Adaptive histogram equalization highlights the features of the first target object, and uses the Shi-Tomasi algorithm to detect high curvature cusps in the extracted edge features;

[0022] A local region of interest is extracted near the high curvature cusp, a first target object is fitted in the local region of interest using the least squares method, and pixels of the fitted first target object are determined in the image, thereby acquiring pixel A.

[0023] Further preferably, the method for acquiring the pixel B and the real-time pixel D of the second target object is as follows:

[0024] Performing distortion removal and noise reduction processing on the collected image of the second target object;

[0025] Adaptive threshold segmentation method is used in the image to extract the highlight area and separate the highlight area from the background; morphological operations are used to remove small noise and fill holes to obtain a coherent highlight area;

[0026] The second target object is extracted by contour detection, the coordinates of the center point of the area where the second target object is located are calculated, and the pixels of the second target object in the image are determined according to the coordinates.

[0027] Further preferably, the method for acquiring the pixel C1 and the real-time pixel C2 of the reference object in the XOY plane is as follows:

[0028] For the image of the reference object obtained, the shape of the reference object in the image is detected by Hough circle transform according to the shape of the reference object and feature extraction is performed. After extraction, the center point of the reference object is obtained, and the pixel of the center point in the image is determined to be the pixel of the reference object in the image.

[0029] Further preferably, the system comprises a first image acquisition device, a second image acquisition device, an image processing module, and a controller, wherein:

[0030] The first image acquisition device and the second image acquisition device are orthogonal to each other and are used to acquire images;

[0031] The image processing module is used to process the first image acquisition device and the second image acquisition device to obtain required pixels;

[0032] The controller is used to calculate the pixel error, and control the robot to move toward the first target object and the second target object with the goal of minimizing the pixel error.

[0033] According to another aspect of the present invention, a robot spatial visual servo control system based on an orthogonal vision system is provided. The system includes an actuator for executing the above-mentioned robot spatial visual servo control method based on an orthogonal vision system.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the robot spatial visual servo control method based on an orthogonal vision system is implemented as described above.

[0035] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0036] 1. The present invention approaches the first target object during XOY direction positioning, then moves in the XYZ plane to approach the second target and further approach the first target, and converts to plane positioning during Z direction positioning instead of using stereo matching to calculate depth. The speed is faster and its accuracy is not limited by the resolution of the image acquisition device, the internal parameter calibration and the observation distance. It can cover a larger working space and directly observe the relative position between the end tool and the target.

[0037] 2. The present invention takes the minimum pixel error as the goal and uses the pixel error to calculate the robot's movement speed. There is no need to reversely infer the three-dimensional position of the spatial point to be positioned from the image information. Therefore, it is less sensitive to the hand-eye calibration error, avoids the cumulative error of pose estimation, improves positioning accuracy, and forms a closed loop of "image intuitive information perception-robot movement", avoiding the segmented delay of "image→pose estimation→motion planning" in traditional methods, and significantly improves the response speed of dynamic scenes.

[0038] 3. In the layout of the present invention in which two cameras are placed orthogonally in space, "orthogonal" means that the optical axes of the two cameras are perpendicular to each other, one observes the XY plane along the Z axis, and the other observes the Z direction along the X axis. This geometric relationship decouples the three-dimensional positioning problem into two independent dimensions: the XY plane camera directly maps the X / Y coordinates of the target through the image, and the Z direction camera converts the depth information into plane information, thereby simplifying the complex three-dimensional coordinate calculation into a step-by-step solution of the plane and depth, which not only avoids the complexity of parallax matching in traditional stereo vision, but also achieves higher accuracy in the Z direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a robot spatial visual servo control method based on an orthogonal visual system constructed according to a preferred embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of the structure of a robot space visual servo control system based on an orthogonal visual system constructed according to a preferred embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of real-time pixel coordinates of a reference object in the camera 1 during operation constructed according to a preferred embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of real-time pixel coordinates of a second target object in camera 2 during operation constructed according to a preferred embodiment of the present invention;

[0043] Figure 5 It is a schematic diagram of the relationship between the movement speed and time of the robot terminal constructed according to the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0045] A robot space visual servo control system based on an orthogonal visual system, the system comprising a first image acquisition device, a second image acquisition device, an image processing module, and a controller, wherein:

[0046] The first image acquisition device and the second image acquisition device are orthogonal to each other and are used to acquire images;

[0047] The image processing module is used to process the first image acquisition device and the second image acquisition device to obtain required pixels;

[0048] The controller is used to calculate the pixel error, and control the robot to move toward the first target object and the second target object with the pixel error being minimized.

[0049] like Figure 1 As shown, in one embodiment of the present invention, the first image acquisition device and the second image acquisition device are respectively a first camera and a second camera, the first camera is arranged in an eye-on-hand manner, that is, it is fixed on the end of the robot, and the second camera is arranged in an eye-outside-hand manner, that is, it is fixed on the experimental table, to form an orthogonal vision system, and the camera's intrinsic parameter calibration and hand-eye calibration are performed respectively. The calibration only needs to be performed once after the installation is completed, and the relative posture of the camera relative to its fixed position will no longer change after calibration.

[0050] In one embodiment of the present invention, the first target object is a probe, the second target object is a high-reflective target, and the reference object is a circular workpiece. The probe, the first camera and the high-reflective target are all arranged at the end of the robot, and the circular workpiece is arranged on the test bench.

[0051] The circular workpiece to be positioned is fixed on the experimental table. The first camera is fixed on the end of the robot and moves synchronously with the end of the robot. The camera plane is parallel to the plane of the experimental table as much as possible, and the workpiece is within the field of view to locate the XY direction. The second camera is fixed on the experimental table. After the system is built, it always remains stationary. The camera plane is perpendicular to the plane of the experimental table as much as possible, and a highly reflective positioning target point is attached within the field of view of the end of the robot to locate the Z direction. The first camera and the second camera form an orthogonal vision system.

[0052] The first camera is placed in the way of eye on hand, and the checkerboard calibration plate is fixed on the plane of the experimental table. The robot terminal posture is changed, and the relationship between the robot base coordinate system and the checkerboard coordinate system remains unchanged. The transformation matrix of the camera coordinate system relative to the checkerboard coordinate system is solved through the checkerboard photos taken by the camera, and the transformation matrix of the robot terminal coordinate system relative to the base coordinate is recorded, so as to solve the transformation matrix of the camera coordinate system relative to the robot terminal coordinate system. The intrinsic calibration and hand-eye calibration of the first camera are completed.

[0053] The second camera is placed with the eye outside the hand, and the checkerboard calibration plate is fixed to the end of the robot. The posture of the robot end is changed, and the relationship between the robot base coordinate system and the checkerboard coordinate system remains unchanged. The transformation matrix of the camera coordinate system relative to the checkerboard coordinate system is solved through the checkerboard photos taken by the camera, and the transformation matrix of the robot end coordinate system relative to the base coordinate system is recorded, thereby solving the transformation matrix of the camera coordinate system relative to the robot base coordinate system. The internal parameter calibration and hand-eye calibration of the second camera are completed.

[0054] Take the hand-eye calibration with eyes on hands as an example:

[0055]

[0056] Converting to AX=XB format, we get:

[0057]

[0058] in, is the transformation matrix of A relative to B. The numerical subscripts represent the relative transformation relationship after the ith movement. Using the Tsai method to solve X is the hand-eye matrix.

[0059] The calibrated internal parameter matrix is ​​K:

[0060]

[0061] Among them, (u 0 ,v 0 ) is the camera optical center, f x ,f y is the camera focal length.

[0062] Before running the visual servo control, the image is collected to determine the image processing operation steps and thresholds for extracting the target image features, and the visual servo matrix that associates the image pixels with the robot terminal speed is derived. When running the spatial visual servo control, the orthogonal camera collects the current image in real time and extracts the image features. The difference between the current image features and the target image features is input into the derived visual servo matrix, and the robot terminal speed is output in combination with the hand-eye calibration matrix.

[0063] First, the robot's XY direction movement is controlled by visual servoing based on the image feature difference collected in real time by the first camera. When the XY direction difference decreases to the preset threshold 1, the second camera visual servoing is added to control the robot's Z direction movement. The movement stops when the XYZ direction difference decreases to the preset threshold 2.

[0064] like Figure 2 As shown, a robot spatial visual servo control method based on an orthogonal vision system specifically includes the following steps:

[0065] S1 collects orthogonal images, performs image processing operations such as dedistortion, denoising and threshold segmentation, and extracts pixels in the target image.

[0066] A positioning probe is fixed at the end of the robot, which has fixed pixels in the field of view of the first camera. Before running the visual servo control, it is necessary to use the first camera and the second camera to collect images and pre-process the images to determine the best method and threshold to extract the image features required in the visual servo process and perform sub-pixel accuracy optimization.

[0067] There are circular artifacts and needle tips in the field of view of the first camera. First, the image is dedistorted according to the calibrated camera internal parameters, and the acquired image is converted into grayscale format. Then, Gaussian blur is applied to smooth the image to reduce noise interference, and then Canny edge detection is used to extract edge features in the image. For the needle tip: adaptive histogram equalization is applied to highlight the features of the needle tip area, and the Shi-Tomasi algorithm is used to detect high curvature cusps in edge points. The local area of ​​interest is extracted near the initially detected needle tip, and the cusp is fitted by the least squares method based on the pixel intensity distribution and gradient direction characteristics.

[0068] For circular artifacts: Detect the circle in the image through Hough circle transform, set appropriate parameters to filter the features to be extracted, and use the gradient descent method near the filtered circle center to fine-tune the circle center coordinates based on the pixel intensity gradient.

[0069] There is a highly reflective target in the field of view of the second camera. First, the image is dedistorted according to the calibrated camera intrinsic parameters, and the captured image is converted into grayscale format. Gaussian blur is used to reduce noise while retaining the edges of the reflective area. Then, the highlight area is extracted using an adaptive threshold segmentation method to separate it from the background. Then, morphological operations are used to remove small noise and fill holes to obtain a coherent highlight area. Subsequently, the target outline is extracted by contour detection, and irrelevant areas are filtered based on the shape and size of the area, retaining only the nearly circular target area. Then, the center point position of the retained area is calculated, and the centroid method is used to calculate the precise center coordinates based on the pixel brightness distribution in the area. In order to further optimize the center point, the local region of interest of the candidate area is extracted, and the optimization of sub-pixel accuracy is achieved by the least squares fitting method.

[0070] The needle tip pixels extracted by the first camera are used as the target image features, and the target point features in the image captured by the second camera when the needle tip touches the center of the workpiece are used as the target image features. The positions of the target image features will not change during the robot positioning process guided by visual servoing in the operating space.

[0071] S2 first controls the robot's XY direction movement based on the image feature difference value collected in real time by the first camera through visual servoing. When the XY direction difference decreases to the preset threshold value 1, the second camera visual servoing is added to control the robot's Z direction movement. When the XYZ direction difference decreases to the preset threshold value 2, the robot stops moving.

[0072] The image is collected in real time and the current image features are extracted. The difference between the current image features and the target image features is input into the derived visual servo matrix to associate the image pixels with the robot terminal velocity.

[0073] The control goal of image-based visual servoing is to minimize the pixel error e(t):

[0074] e(t)=ss *

[0075] Among them, s is the current image feature vector in the image plane, s * is the target image feature vector. Design the speed controller, establish the relationship matrix between s and the camera speed, and define the speed vector as:

[0076] v c =(v c ,ω c )

[0077]

[0078] Among them, L is the visual servo matrix, and the relationship between the error e and the camera velocity vector is expressed as:

[0079]

[0080] Make the error e decay exponentially ( λ is a constant), the robot terminal velocity can be obtained:

[0081] v e =-Adj(T)L + λe

[0082] Wherein, T is the hand-eye matrix obtained by calibration in step S1, Adj(T) is the velocity adjoint matrix, and L + is the Moore-Penrose pseudoinverse of L.

[0083] According to the pinhole imaging principle, the transformation relationship of the spatial point (Xc, Yc, Zc) from the camera coordinate system to the image coordinate system is:

[0084]

[0085] The conversion relationship between the image pixel coordinate system and the image physical coordinate system is:

[0086]

[0087] where d x and d y Represents the physical size of each pixel in the x and y directions respectively. According to the calibrated intrinsic parameters, We can get:

[0088]

[0089] Take the camera in the hand as an example, when the space point is fixed and the camera is moved, the speed relationship is:

[0090]

[0091] We can get:

[0092]

[0093] From this, the visual servo matrix L can be derived:

[0094]

[0095] Among them, (u, v) is the feature pixel of the current image, Z C is the spatial depth of the current image feature, which has little effect on the calculation and can be set to a constant.

[0096] During the visual servoing process, the current image is collected in real time by two orthogonally placed cameras. According to the proposed image feature extraction method, the first camera extracts the central pixel point of the workpiece as the current image feature, and the difference between it and the needle tip feature pixel point is input into the visual servo feature matrix; the second camera extracts the central pixel point of the highly reflective target as the current image feature, and the difference between it and the target target feature pixel point is input into the visual servo feature matrix. The matrix associates the image pixel space with the robot Cartesian space. By inputting the real-time collected image, the image features are extracted, and the robot terminal velocity is directly output to guide the robot positioning.

[0097] S3 Considering safety, we first do not control the robot's downward movement in the Z direction, but only control the robot to move parallel to the experimental table in the XY direction. According to the image feature difference collected in real time by the first camera, the robot is controlled to move in the XY direction by visual servo until the distance between the current image feature pixel and the target image pixel is reduced to the preset threshold 1. Then, the second camera is added to control the robot's movement in the Z direction until the distance between the current image feature pixel and the target image pixel of the first camera and the second camera is reduced to the preset threshold 2 and stops moving. At this time, it is considered that the visual servo control is over and the robot has been guided to complete positioning.

[0098] Based on the KUKA LBR iiwa robot and the KUKA FRI real-time interface, the ROS package combined with the calibrated hand-eye matrix is ​​used to perform inverse kinematics calculations on the robot to achieve real-time control of the terminal velocity.

[0099] The entire experimental scene setting constitutes a set of spatial visual servo robot control systems based on orthogonal vision systems. The communication between modules is realized based on the ROS robot system, and the algorithm runs in multiple processes on the host computer Linux real-time system. The hardware platform is developed based on the KUKA LBR iiwa robot. Based on the KUKA FRI real-time interface, the software and hardware platforms are connected to perform real-time control of robot visual servo.

[0100] The present invention realizes high-precision spatial positioning of a target by guiding a robot using spatial visual servoing based on an orthogonal vision system, and completes a verification experiment with a spatial positioning accuracy of 0.05 mm on KUKA iiwa.

[0101] The verification process is as follows: This experiment uses a robot and two orthogonally placed industrial cameras. The cameras collect images in real time and extract the pixel coordinates of image features. The derived visual servo formula is imported to calculate the robot terminal speed, such as Figure 3-5 As shown, Figure 3 This is a schematic diagram of the real-time pixel coordinates of the reference object in camera 1 during operation. Figure 4 This is a schematic diagram of the real-time pixel coordinates of the second target object in camera 2 during operation. As the reference object gradually approaches the target pixel point, the robot speed slows down and the coordinate points become more concentrated near the target point. Figure 5 This is a schematic diagram of the relationship between the robot's terminal motion speed and time. The speed is getting slower and slower, and the speed in the XY direction first appears, and after it drops to a certain threshold, the Z direction begins to have speed.

[0102] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A robot spatial visual servo control method based on an orthogonal vision system, characterized in that: The method comprises the following steps: Collecting images of the first target object and the second target object respectively, and performing image processing on the collected images to obtain pixels A of the first target object and pixels B of the second target object respectively; The robot moves in the XOY plane, collects the image of the reference object in real time and obtains the pixel C1 of the reference object in the XOY plane, and controls the robot to approach the first target object in the XOY plane with the goal of minimizing the pixel error between the pixel C1 of the reference object in the XOY plane and the pixel A of the first target object until the distance between the two meets a preset first threshold; The robot moves in the XYZ plane, collects the image of the reference object and the image of the second target object in real time and obtains the real-time pixels of the reference object and the second target in the image, which are the real-time pixel C2 of the reference object and the real-time pixel D of the second target, respectively. The pixel error between the real-time pixel C2 of the reference object and the pixel A of the first target, and the pixel error between the real-time pixel D and the pixel B of the second target are minimized, respectively. The robot is controlled to approach the first target object and the second target object in the XYZ plane until a preset second threshold is met, thereby realizing the control of the robot.

2. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1, characterized in that: When the robot is controlled to approach the first target in the XOY plane and the robot is controlled to approach the first target and the second target in the XYZ plane, the movement speed of the robot is calculated according to the following formula: v e =-Adj(T)L + λe Among them, v e is the robot terminal velocity, T is the hand-eye matrix of the first image acquisition device or the second image acquisition device obtained by calibration, Adj(T) is the velocity adjoint matrix, L + is the Moore-Penrose pseudo-inverse of L, λ is a constant used to control the relative velocity of the robot, L is the visual servo matrix, and e is the pixel error.

3. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 2, characterized in that: The visual servo matrix L is as follows: Wherein, (u, v) is the real-time coordinate of the pixel point in the image of the reference object or the second target object at the current moment, (u0, v0) is the coordinate of the optical center of the first image acquisition device or the second image acquisition device, and f x ,f y is the focal length of the first image acquisition device or the second image acquisition device along the X direction and the Y direction, Z C It is the spatial depth of the image of the reference object or the second target object at the current moment.

4. A robot spatial visual servo control method based on an orthogonal visual system as claimed in claim 3, characterized in that: The formula for the pixel error is as follows: e(t)=s-s * Where s is the real-time pixel C1 of the reference object in the XOY plane, the real-time pixel C2 of the reference object or the real-time pixel D of the second target object, s * is pixel A of the first object or pixel B of the second object, and t is the time.

5. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1 or 4, characterized in that: The method for acquiring the pixel A of the first target object is as follows: Performing dedistortion and noise reduction processing on the collected image of the first target object, and extracting edge features in the image; Adaptive histogram equalization highlights the features of the first target object, and uses the Shi-Tomasi algorithm to detect high curvature cusps in the extracted edge features; A local region of interest is extracted near the high curvature cusp, a first target object is fitted in the local region of interest using the least squares method, and pixels of the fitted first target object are determined in the image, thereby acquiring pixel A.

6. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1 or 4, characterized in that: The method for acquiring the pixel B and the real-time pixel D of the second target object is as follows: Performing distortion removal and noise reduction processing on the collected image of the second target object; Using an adaptive threshold segmentation method in the image to extract a highlight area and separate the highlight area from the background; Use morphological operations to remove small noise and fill holes to obtain coherent highlight areas; The second target object is extracted by contour detection, the coordinates of the center point of the area where the second target object is located are calculated, and the pixels of the second target object in the image are determined according to the coordinates.

7. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1 or 4, characterized in that: The method for acquiring the pixel C1 and the real-time pixel C2 of the reference object in the XOY plane is as follows: For the image of the reference object obtained, the shape of the reference object in the image is detected by Hough circle transform according to the shape of the reference object and feature extraction is performed. After extraction, the center point of the reference object is obtained, and the pixel of the center point in the image is determined to be the pixel of the reference object in the image.

8. A robot spatial visual servo control system based on an orthogonal vision system according to any one of claims 1 to 7, characterized in that: The system comprises a first image acquisition device, a second image acquisition device, an image processing module, and a controller, wherein: The first image acquisition device and the second image acquisition device are orthogonal to each other and are used to acquire images; The image processing module is used to process the first image acquisition device and the second image acquisition device to obtain required pixels; The controller is used to calculate the pixel error, and control the robot to move toward the first target object and the second target object with the goal of minimizing the pixel error.

9. A robot spatial visual servo control system based on an orthogonal vision system, characterized in that: The system comprises an actuator, which is used to execute the robot spatial visual servo control method based on an orthogonal vision system as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot spatial visual servo control method based on an orthogonal vision system described in any one of claims 1 to 7 is implemented.

Images