A robot space vision servo control method and system based on an orthogonal vision system

Abstract

The application belongs to the technical field of robot control, and discloses a robot space vision servo control method and system based on an orthogonal vision system. The method comprises the following steps: acquiring images of a first target object and a second target object to obtain pixels A of the first target object and pixels B of the second target object; taking the minimum pixel error between the pixel C1 of a reference object in the XOY plane and the pixel A of the first target object as the target, controlling the robot to approach the first target object in the XOY plane until the distance between the two satisfies a preset first threshold; taking the minimum pixel error between the real-time pixel C2 of the reference object and the pixel A of the first target object and the pixel error between the real-time pixel D of the second target and the pixel B as the target, controlling the robot to approach the first target object and the second target object in the XYZ plane until a preset second threshold is satisfied, thereby realizing the control of the robot. Through the application, high-precision spatial positioning of the target is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of robot control, and more particularly relates to a robot space vision servo control method and system based on an orthogonal vision system. BACKGROUND

[0002] With the continuous development of manufacturing industry, the demand for high-precision robot processing is increasing. The number of processing tasks is rapidly increasing, while the requirements for processing quality are becoming more and more strict. Among numerous robot control technologies, the vision servo system has become a key solution for realizing high-precision space positioning and operation due to its flexibility, high efficiency and robustness in complex environments.

[0003] The vision servo system is mainly divided into position-based visual servo (PBVS) and image-based visual servo (IBVS). PBVS input is calculated in three-dimensional space by estimating the position of image features in three-dimensional space to control robot motion, which is very sensitive to calibration error and environmental changes. IBVS directly calculates the control input in two-dimensional image space by adjusting the position of the target in the image to control the robot motion, which has the advantages of high robustness to calibration error, simple calculation, strong real-time performance, and is particularly suitable for dynamic and complex environment tasks.

[0004] According to the installation mode of the camera, the vision servo system is divided into eye-in-hand and eye-to-hand. The camera of eye-in-hand can dynamically adjust the field of view to observe the target object, which is suitable for realizing high spatial positioning accuracy, but it is difficult to directly observe the relative position between the end tool and the target, and the hand-eye calibration accuracy is required. The camera of eye-to-hand is fixed in the workspace, which can observe the target and the robot end at the same time, and is suitable for multi-robot cooperation and complex workpiece processing tasks, and can cover a larger workspace, but its accuracy is limited by the resolution and observation distance of the camera. Therefore, a method is needed to solve the above problems of eye-in-hand and eye-to-hand. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a robot space vision servo control method and system based on an orthogonal vision system, which solves the problems of high requirements for camera intrinsic parameter calibration and hand-eye calibration accuracy, poor real-time performance, and XY direction accuracy limited by the resolution of the camera, and Z direction accuracy limited by the observation distance of the camera.

[0006] To achieve the above purpose, according to one aspect of the present application, a robot space vision servo control method based on an orthogonal vision system is provided, which comprises the following steps:

[0007] Acquire images of the first target and the second target respectively, and perform image processing on the acquired images to obtain pixels A of the first target and pixels B of the second target respectively;

[0008] The robot moves in the XOY plane, captures an image of a reference object in real time, and obtains a pixel C1 of the reference object in the XOY plane. With the goal of minimizing the pixel error between the pixel C1 of the reference object in the XOY plane and the pixel A of the first target object, the robot is controlled to approach the first target object in the XOY plane until the distance between the two meets a preset first threshold.

[0009] The robot moves in the XYZ plane, collects images of the reference object and the second target object in real time, and obtains real-time pixels of the reference object and the second target object in the image, which are respectively the real-time pixel C2 of the reference object and the real-time pixel D of the second target. With the goal of minimizing the pixel error between the real-time pixel C2 of the reference object and the pixel A of the first target, and the pixel error between the real-time pixel D and the pixel B of the second target, the robot is controlled to approach the first target object and the second target object in the XYZ plane until a preset second threshold is met, thereby realizing the control of the robot.

[0010] Further preferably, when the robot is controlled to approach the first target in the XOY plane and when the robot is controlled to approach the first target and the second target in the XYZ plane, the movement speed of the robot is calculated according to the following formula:

[0011] v e =-Adj(T)L + λe

[0012] Among them, v e is the robot terminal velocity, T is the hand-eye matrix of the first image acquisition device or the second image acquisition device obtained by calibration, Adj(T) is the velocity adjoint matrix, L + is the Moore-Penrose pseudo-inverse of L, λ is a constant used to control the relative velocity of the robot, L is the visual servo matrix, and e is the pixel error.

[0013] Further preferably, the visual servo matrix L is as follows:

[0014]

[0015] Wherein, (u, v) is the real-time coordinate of the pixel point in the image of the reference object or the second target object at the current moment, (u0, v0) is the coordinate of the optical center of the first image acquisition device or the second image acquisition device, and f x ,f y is the focal length of the first image acquisition device or the second image acquisition device along the X direction and the Y direction, Z CIt is the spatial depth of the image of the reference object or the second target object at the current moment.

[0016] Further preferably, the formula for the pixel error is as follows:

[0017] e(t)=ss *

[0018] Where s is the pixel C1 of the reference object in the XOY plane, the real-time pixel C2 of the reference object or the real-time pixel D of the second target object, s * is pixel A of the first object or pixel B of the second object, and t is the time.

[0019] Further preferably, the method for acquiring the pixel A of the first target object is as follows:

[0020] Performing smoothing and noise reduction processing on the collected image of the first target object and extracting edge features in the image;

[0021] Adaptive histogram equalization highlights the features of the first target object, and uses the Shi-Tomasi algorithm to detect high curvature cusps in the extracted edge features;

[0022] A local region of interest is extracted near the high curvature cusp, a first target object is fitted in the local region of interest using the least squares method, and pixels of the fitted first target object are determined in the image, thereby acquiring pixel A.

[0023] Further preferably, the method for acquiring the pixel B and the real-time pixel D of the second target object is as follows:

[0024] Performing dedistortion and noise reduction processing on the collected image of the second target object;

[0025] Adaptive threshold segmentation is used to extract highlight areas in the image and separate them from the background. Morphological operations are used to remove small noise and fill holes to obtain coherent highlight areas.

[0026] The second target object is extracted by contour detection, the coordinates of the center point of the area where the second target object is located are calculated, and the pixels of the second target object in the image are determined according to the coordinates.

[0027] Further preferably, the method for acquiring the pixel C1 and the real-time pixel C2 of the reference object in the XOY plane is as follows:

[0028] For the image of the reference object obtained, the shape of the reference object in the image is detected by Hough circle transform according to the shape of the reference object and feature extraction is performed. After extraction, the center point of the reference object is obtained, and the pixel of the center point in the image is determined to be the pixel of the reference object in the image.

[0029] Further preferably, the system includes a first image acquisition device, a second image acquisition device, an image processing module, and a controller, wherein:

[0030] The first image acquisition device and the second image acquisition device are orthogonal to each other and are used to acquire images;

[0031] The image processing module is used to process the first image acquisition device and the second image acquisition device to obtain required pixels;

[0032] The controller is used to calculate the pixel error and control the robot to move toward the first target object and the second target object with the pixel error being minimized.

[0033] According to another aspect of the present invention, a robot spatial visual servo control system based on an orthogonal vision system is provided. The system includes an actuator for executing the above-mentioned robot spatial visual servo control method based on an orthogonal vision system.

[0034] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the robot spatial visual servo control method based on the orthogonal vision system is implemented.

[0035] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0036] 1. The present invention approaches a first target during XOY positioning, then moves in the XYZ plane to approach a second target and further approach the first target. During Z positioning, the positioning is converted to plane positioning instead of using stereo matching to calculate depth. This makes the positioning faster and its accuracy is not limited by the resolution of the image acquisition device, the internal parameter calibration, and the observation distance. It can cover a larger workspace and directly observe the relative position between the end tool and the target.

[0037] 2. The present invention aims to minimize pixel error and uses pixel error to calculate the robot's motion speed. There is no need to reversely infer the three-dimensional position of the spatial point to be positioned from the image information. Therefore, it is less sensitive to hand-eye calibration errors, avoids the cumulative error of pose estimation, improves positioning accuracy, and forms a closed loop of "image intuitive information perception-robot motion", avoiding the segmented delay of "image → pose estimation → motion planning" in traditional methods, and significantly improves the response speed of dynamic scenes.

[0038] 3. In the present invention's layout of two cameras placed orthogonally in space, "orthogonal" means that their optical axes are perpendicular to each other. One camera observes the XY plane along the Z axis, and the other observes the Z direction along the X axis. This geometric relationship decouples the three-dimensional positioning problem into two independent dimensions: the XY plane camera directly maps the target's X / Y coordinates through images, while the Z direction camera converts depth information into plane information, thereby simplifying complex three-dimensional coordinate calculations into a step-by-step solution of plane and depth. This avoids the complexity of parallax matching in traditional stereo vision while achieving higher accuracy in the Z direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a robot spatial visual servo control method based on an orthogonal vision system constructed according to a preferred embodiment of the present invention;

[0040] Figure 2 Schematic diagram of the structure of a robot spatial visual servo control system based on an orthogonal vision system constructed according to a preferred embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of the real-time pixel coordinates of a reference object in the camera 1 during operation constructed according to a preferred embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of the real-time pixel coordinates of the second target object in the camera 2 during operation constructed according to the preferred embodiment of the present invention;

[0043] Figure 5 It is a schematic diagram of the relationship between the movement speed of the robot terminal and time constructed according to the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0045] A robot spatial visual servo control system based on an orthogonal vision system, the system comprising a first image acquisition device, a second image acquisition device, an image processing module, and a controller, wherein:

[0046] The first image acquisition device and the second image acquisition device are orthogonal to each other and are used to acquire images;

[0047] The image processing module is used to process the first image acquisition device and the second image acquisition device to obtain required pixels;

[0048] The controller is used to calculate the pixel error and control the robot to move toward the first target object and the second target object with the pixel error being minimized.

[0049] like Figure 1 As shown, in one embodiment of the present invention, the first image acquisition device and the second image acquisition device are respectively the first camera and the second camera. The first camera is set in an eye-on-hand manner, that is, it is fixed on the end of the robot, and the second camera is set in an eye-outside-hand manner, that is, it is fixed on the experimental table, forming an orthogonal vision system, and performing intrinsic parameter calibration of the camera and hand-eye calibration respectively. The calibration only needs to be performed once after the construction is completed. After calibration, the relative position of the camera relative to its fixed position no longer changes.

[0050] In one embodiment of the present invention, the first target is a probe, the second target is a highly reflective target, and the reference object is a circular workpiece. The probe, the first camera, and the highly reflective target are all arranged at the end of the robot, and the circular workpiece is arranged on the test bench.

[0051] The circular workpiece to be positioned is fixed on the experimental table. The first camera is fixed on the end of the robot and moves synchronously with the end of the robot. The camera plane is as parallel to the plane of the experimental table as possible, and the workpiece is ensured to be within the field of view to locate the XY direction. The second camera is fixed on the experimental table and remains stationary after the system is set up. The camera plane is as perpendicular to the plane of the experimental table as possible, and a highly reflective positioning target point is attached within the field of view of the end of the robot to locate the Z direction. The first camera and the second camera form an orthogonal vision system.

[0052] The first camera is positioned with the eye in the hand. A checkerboard calibration plate is fixed on the experimental table. The robot's end-point posture is transformed, and the relationship between the robot's base coordinate system and the checkerboard coordinate system remains unchanged. The camera's checkerboard images are used to calculate the transformation matrix of the camera coordinate system relative to the checkerboard coordinate system. The transformation matrix of the robot's end-point coordinate system relative to the base coordinate system is recorded, thereby solving the transformation matrix of the camera coordinate system relative to the robot's end-point coordinate system. This completes the intrinsic parameter calibration and hand-eye calibration of the first camera.

[0053] The second camera is placed with the eye outside the hand. A checkerboard calibration plate is fixed to the robot's end-point. The relationship between the robot's base coordinate system and the checkerboard coordinate system remains unchanged when the robot's end-point pose is changed. The camera's checkerboard images are used to calculate the transformation matrix of the camera's coordinate system relative to the checkerboard coordinate system. The transformation matrix of the robot's end-point coordinate system relative to the base coordinate system is recorded, thereby calculating the transformation matrix of the camera's coordinate system relative to the robot's base coordinate system. This completes the intrinsic parameter calibration and hand-eye calibration of the second camera.

[0054] Take the hand-eye calibration with the eye on the hand as an example:

[0055]

[0056] Converting to AX=XB format, we get:

[0057]

[0058] in, is the transformation matrix of A relative to B. The numerical subscripts represent the relative transformation relationship after the i-th movement. Using the Tsai method, solve for X to obtain the hand-eye matrix.

[0059] The internal parameter matrix obtained by calibration is K:

[0060]

[0061] Among them, (u0,v0) is the camera optical center, f x ,f y is the camera focal length.

[0062] Before executing visual servo control, images are captured to determine the image processing steps and thresholds for extracting target image features. This process then derives a visual servo matrix that relates image pixels to the robot's end-point velocity. During spatial visual servo control, an orthogonal camera captures the current image in real time and extracts image features. The difference between the current and target image features is input into the derived visual servo matrix, which is then combined with the hand-eye calibration matrix to output the robot's end-point velocity.

[0063] First, the robot's XY direction movement is controlled by visual servoing based on the image feature difference collected in real time by the first camera. When the XY direction difference decreases to the preset threshold 1, the second camera visual servoing is added to control the robot's Z direction movement. The movement is stopped when the XYZ direction difference decreases to the preset threshold 2.

[0064] like Figure 2 As shown, a robot spatial visual servo control method based on an orthogonal vision system specifically includes the following steps:

[0065] S1 collects orthogonal images and performs image processing operations such as dedistortion, denoising and threshold segmentation to extract pixels in the target image.

[0066] A positioning probe is fixed at the end of the robot, with a fixed pixel range within the field of view of the first camera. Before running visual servo control, images are captured using the first and second cameras. These images are then preprocessed to determine the best method and threshold for extracting the image features required for visual servoing, and then optimized for sub-pixel accuracy.

[0067] The first camera's field of view contains a circular artifact and a needle tip. The image is first dedistorted based on the calibrated camera intrinsic parameters and converted to grayscale. Gaussian blur is then applied to smooth the image to reduce noise, and Canny edge detection is then used to extract edge features. Regarding the needle tip, adaptive histogram equalization is applied to highlight the needle tip region. High-curvature cusps are detected among the edge points using the Shi-Tomasi algorithm. A local region of interest is extracted near the initially detected needle tip. The cusp is then fitted using the least squares method based on the pixel intensity distribution and gradient direction characteristics.

[0068] For circular artifacts: Detect circles in the image using Hough circle transform, set appropriate parameters to filter out the features to be extracted, and use the gradient descent method near the filtered circle center to fine-tune the circle center coordinates based on the pixel intensity gradient.

[0069] A highly reflective target is present in the second camera's field of view. The image is first dedistorted based on the calibrated camera intrinsics and converted to grayscale. Gaussian blurring is then used to reduce noise while preserving the edges of the reflective areas. Adaptive threshold segmentation is then used to extract highlight regions and separate them from the background. Morphological operations are then used to remove small noise and fill holes to obtain a coherent highlight region. Contour detection is then used to extract the target outline, and irrelevant regions are filtered based on their shape and size, retaining only the nearly circular target region. The center point of the retained region is then calculated, and the centroid method is used to calculate the precise center coordinates based on the pixel brightness distribution within the region. To further optimize the center point, a local region of interest (ROI) is extracted from the candidate region, and sub-pixel accuracy is achieved through least squares fitting.

[0070] The needle tip pixels extracted by the first camera are used as the target image features, and the target point features in the image captured by the second camera when the needle tip touches the center of the workpiece are used as the target image features. The positions of the target image features will not change during the robot positioning process guided by visual servoing in the running space.

[0071] S2 first controls the robot's XY motion based on the image feature difference collected in real time by the first camera. When the XY difference decreases to a preset threshold value 1, the second camera is added to control the robot's Z motion. When the XYZ difference decreases to a preset threshold value 2, the robot stops moving.

[0072] The image is collected in real time and the current image features are extracted. The difference between the current image features and the target image features is input into the derived visual servo matrix to associate the image pixels with the robot terminal velocity.

[0073] The control goal of image-based visual servoing is to minimize the pixel error e(t):

[0074] e(t)=ss *

[0075] Among them, s is the current image feature vector in the image plane, s * is the target image feature vector. Design the speed controller, establish the relationship matrix between s and camera speed, and define the speed vector as:

[0076] v c =(v c ,ω c )

[0077]

[0078] Where L is the visual servo matrix, and the relationship between the error e and the camera velocity vector is expressed as:

[0079]

[0080] Make the error e decay exponentially ( λ is a constant), the robot terminal velocity can be obtained as:

[0081] v e =-Adj(T)L + λe

[0082] Wherein, T is the hand-eye matrix obtained by calibration in step S1, Adj(T) is the velocity adjoint matrix, and L + is the Moore-Penrose pseudoinverse of L.

[0083] According to the pinhole imaging principle, the transformation relationship of the spatial point (Xc, Yc, Zc) from the camera coordinate system to the image coordinate system is:

[0084]

[0085] The conversion relationship between the image pixel coordinate system and the image physical coordinate system is:

[0086]

[0087] where d x and d y Represents the physical size of each pixel in the x and y directions respectively. According to the calibrated internal parameters, We can get:

[0088]

[0089] Taking the camera in hand as an example, the speed relationship of a fixed spatial point and moving the camera is:

[0090]

[0091] The result is:

[0092]

[0093] The visual servoing matrix L can be derived as:

[0094]

[0095] where (u, v) is the current image feature pixel, Z C is the spatial depth of the current image feature, which has little effect on the calculation and can be set as a constant.

[0096] During the running of the visual servoing process, the current image is collected in real time by two cameras placed orthogonally. According to the proposed image feature extraction method, the first camera extracts the center pixel of the workpiece as the current image feature, and the difference between it and the needle tip feature pixel is input into the visual servoing feature matrix. The second camera extracts the center pixel of the high-reflective target as the current image feature, and the difference between it and the target feature pixel is input into the visual servoing feature matrix. This matrix relates the image pixel space to the robot Cartesian space, and by inputting the real-time collected image, extracting the image feature, and directly outputting the robot end speed, the robot positioning is guided.

[0097] S3 Considering safety, we first do not control the robot Z direction descending motion, only control the robot XY direction parallel to the experimental table motion. According to the image feature difference value visual servoing control of the first camera, the robot XY direction motion is controlled until the distance between the current image feature pixel and the target image pixel is reduced to the preset threshold 1. Then the second camera visual servoing control of the robot Z direction motion is added until the distance between the current image feature pixels of the first camera and the second camera and the target image pixel is reduced to the preset threshold 2, and the motion is stopped. At this time, it is considered that the visual servoing control is completed, and the robot positioning is guided.

[0098] Based on the KUKA LBR iiwa robot and the KUKA FRI real-time interface, the inverse kinematics calculation of the robot is realized by using the ROS package combined with the calibrated hand-eye matrix to realize the real-time control of the end speed.

[0099] The entire experimental scene setting constitutes a set of spatial visual servoing robot control system based on the orthogonal visual system. Based on the ROS robot system, the communication between modules is realized, the algorithm is run in multiple processes on the host computer Linux real-time system, the hardware platform is developed based on the KUKA LBR iiwa robot, based on the KUKA FRI real-time interface, the software and hardware platforms are connected, and the real-time control of the robot visual servoing is realized.

[0100] The present invention realizes the spatial visual servo-guided robot to achieve high-precision spatial positioning of the target based on the orthogonal vision system, and completes the verification experiment of spatial positioning accuracy of 0.05mm on KUKA iiwa.

[0101] The verification process is as follows: This experiment uses a robot and two orthogonally placed industrial cameras. The cameras collect images in real time and extract the pixel coordinates of image features. The derived visual servo formula is imported to calculate the robot terminal speed, such as Figures 3-5 As shown, Figure 3 This is a schematic diagram of the real-time pixel coordinates of the reference object in camera 1 during operation. Figure 4 This is a schematic diagram of the real-time pixel coordinates of the second target object in camera 2 during operation. As the reference object gradually approaches the target pixel point, the robot speed slows down and the coordinate points become more concentrated near the target point. Figure 5 This is a diagram of the relationship between the robot's terminal motion speed and time. The speed is getting slower and slower, and the speed in the XY direction first increases, and after it drops to a certain threshold, the Z direction begins to increase.

[0102] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A robot spatial visual servo control method based on an orthogonal vision system, characterized in that: The method comprises the following steps: S1 collects orthogonal images through the first camera and the second camera, performs dedistortion, denoising and threshold segmentation image processing operations, and extracts pixels in the target image; The first camera is set up in an eye-on-hand manner, i.e., fixed to the end of the robot, and the second camera is set up in an eye-out-of-hand manner, i.e., fixed to the test bench, to form an orthogonal vision system and complete the hand-eye calibration of the first and second cameras. Specifically, a first target object and a second target object are set up at the end of the robot, and a reference object is set up on the test bench. The first target object and the reference object are in the field of view of the first camera, and the second target object is in the field of view of the second camera. S2: First, the robot is visually servoed to move in the X and Y directions based on the difference in image features collected in real time by the first camera. When the difference in the X and Y directions decreases to a preset first threshold, the second camera is added to visually servo the robot in the Z direction. When the difference in the X, Y and Z directions decreases to a preset second threshold, the robot stops moving. Specifically, images of the first target object and the second target object are respectively captured to obtain pixel A of the first target object and pixel B of the second target object; First, the robot is controlled to move in the XOY plane, capturing an image of a reference object in real time and obtaining real-time pixels C1 of the reference object. With the goal of minimizing the pixel error between the real-time pixels C1 of the reference object and the pixels A of the first target object, the robot is controlled to approach the first target object in the XOY plane until the distance between the two meets a preset first threshold. Secondly, the robot is controlled to move along the Z direction, and an image of the reference object and an image of the second target object are captured in real time to obtain real-time pixels C2 of the reference object and real-time pixels D of the second target. With the goal of minimizing the pixel error between the real-time pixel C2 of the reference object and pixel A of the first target, and the pixel error between the real-time pixel D and pixel B of the second target, respectively, the robot is controlled to approach the first target and the second target in the XYZ space until a preset second threshold is met, thereby achieving control of the robot; When implementing robot control, the robot's movement speed is calculated according to the following formula: in, is the robot terminal velocity, T is the hand-eye matrix obtained after calibration in step S1, Adj(T) is the velocity adjoint matrix, yes The Moore-Penrose pseudo-inverse of , λ is a constant used to control the relative velocity of the robot, is the visual servo matrix, e is the pixel error; Visual servo matrix as follows: in, is the feature pixel of the current image, is the spatial depth of the current image feature, is the coordinate of the camera's optical center, is the focal length of the camera along the X and Y directions.

2. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1, characterized in that: The formula for the pixel error is as follows: Among them, s is the current image feature vector, is the target image feature vector, and t is the time instant.

3. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1 or 2, characterized in that: The method for acquiring the pixel A of the first target object is as follows: Performing dedistortion and noise reduction processing on the collected image of the first target object, and extracting edge features in the image; Adaptive histogram equalization highlights the features of the first target object, and uses the Shi-Tomasi algorithm to detect high curvature cusps in the extracted edge features; A local region of interest is extracted near the high curvature cusp, a first target object is fitted in the local region of interest using the least squares method, and pixels of the fitted first target object are determined in the image, thereby acquiring pixel A.

4. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1 or 2, characterized in that: The method for acquiring the pixel B and the real-time pixel D of the second target object is as follows: Performing dedistortion and noise reduction processing on the collected image of the second target object; Using adaptive threshold segmentation method to extract highlight area in the image and separate the highlight area from the background; Use morphological operations to remove small noise and fill holes to obtain coherent highlight areas; The second target object is extracted by contour detection, the coordinates of the center point of the area where the second target object is located are calculated, and the pixels of the second target object in the image are determined according to the coordinates.

5. A robot spatial visual servo control method based on an orthogonal vision system as claimed in claim 1 or 2, characterized in that: The method for obtaining the real-time pixel C1 and the real-time pixel C2 of the reference object is as follows: For the image of the reference object obtained, the shape of the reference object in the image is detected by Hough circle transform according to the shape of the reference object and feature extraction is performed. After extraction, the center point of the reference object is obtained, and the pixel of the center point in the image is determined to be the pixel of the reference object in the image.

6. A control system for a robot spatial visual servo control method based on an orthogonal vision system according to any one of claims 1 to 5, characterized in that: The system includes a first image acquisition device, a second image acquisition device, an image processing module, and a controller, wherein: The first image acquisition device and the second image acquisition device are orthogonal to each other and are used to acquire images; The image processing module is used to process the first image acquisition device and the second image acquisition device to obtain required pixels; The controller is used to calculate the pixel error and control the robot to move toward the first target object and the second target object with the pixel error being minimized.

7. A robot spatial visual servo control system based on an orthogonal vision system, characterized in that: The system includes an actuator, which is used to execute the robot spatial visual servo control method based on an orthogonal vision system as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot spatial visual servo control method based on an orthogonal vision system according to any one of claims 1 to 5 is implemented.

Images