Wheel-legged robot control method, device, equipment and readable storage medium

Abstract

This application relates to a control method, apparatus, computer device, computer-readable storage medium, and computer program product for a wheeled robot, applicable in the field of robotics. The method includes: controlling the wheeled robot to rotate based on image information of a positioning marker in an image captured by a camera, and determining a first distance between the camera and the positioning marker; controlling the wheeled robot to move based on the first distance information and a preset distance between the camera and the positioning marker, and determining an angle between the wheeled robot and a preset reference plane based on the image information of the positioning marker in the captured image; controlling the wheeled robot to rotate based on the angle information, and determining a second distance between the camera and the positioning marker based on the image information of the positioning marker; and controlling the wheeled robot to move to a preset position based on the second distance information. This method can improve the accuracy of wheeled robot control.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a control method, apparatus, device, and computer-readable storage medium for a wheeled robot. Background Technology

[0002] With the development of robotics technology, the demand for positioning and attitude control of wheeled robots in applications such as intelligent automatic repositioning and automatic recharging is becoming increasingly prominent. Due to their inherent balance control strategies and wheel slippage characteristics, wheeled robots are prone to uncontrollable attitude changes during frequent turning and positioning. Therefore, how to achieve positioning and attitude control of wheeled robots has become an important research direction.

[0003] Traditional technologies typically involve adding multiple auxiliary sensors or complex structures to achieve positioning and attitude control of wheeled robots. However, various sensors are susceptible to interference under different working conditions, resulting in data discrepancies and data conflicts. This not only increases the complexity of software algorithms but also introduces more uncertainties. In particular, for wheeled robots equipped with only a single camera, the accuracy of wheeled robot control is relatively low. Summary of the Invention

[0004] Therefore, it is necessary to provide a wheeled robot control method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of wheeled robot control in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a control method for a wheeled legged robot. The method includes:

[0006] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0007] Based on the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and the angle information between the wheeled robot and the preset reference plane is determined based on the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0008] Based on the angle information, the wheeled robot is controlled to rotate, and based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0009] Based on the second distance information, the wheeled robot is controlled to move to a preset position.

[0010] In one embodiment, controlling the movement of the wheeled robot based on the first distance information and the preset distance information between the camera and the positioning marker includes:

[0011] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane is determined;

[0012] The preset distance information is determined based on the included angle information and the lateral distance information between the preset position and the center point of the positioning mark;

[0013] Based on the first distance information, the wheeled robot is controlled to move until the first distance information is within the preset distance range corresponding to the preset distance information.

[0014] In one embodiment, determining the first distance information between the camera and the positioning identifier includes:

[0015] Based on the image information of the positioning marker in the image captured by the camera of the wheeled robot, the basic first distance information between the camera and the positioning marker is determined;

[0016] If the basic first distance information is greater than a preset distance threshold, multiple candidate first distance information between the camera and the positioning marker are determined based on the image information of the positioning marker in the image captured by the camera of the wheeled robot.

[0017] The first distance information is determined based on the average information among the plurality of candidate first distance information;

[0018] The step of determining the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane based on the image information of the positioning mark in the image captured by the camera of the wheeled robot includes:

[0019] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, determine multiple candidate angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane;

[0020] The included angle information is determined based on the average information among the multiple candidate included angle information.

[0021] In one embodiment, after controlling the wheeled robot to move to a preset position based on the second distance information, the method further includes:

[0022] Obtain the current abscissa of the center point of the positioning marker in the image captured by the camera of the wheeled robot;

[0023] Based on the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning mark, determine whether the wheeled robot has successfully moved to the preset position.

[0024] In one embodiment, controlling the rotation of the wheeled robot based on image information of positioning markers in images captured by the robot's camera includes:

[0025] Image recognition processing is performed on the images captured by the camera of the wheeled robot to obtain the image information of the positioning marker in the images captured by the camera of the wheeled robot;

[0026] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the lateral deviation information between the center point of the positioning mark and the center point of the image captured by the camera of the wheeled robot is determined.

[0027] Based on the lateral deviation information, the wheeled robot is controlled to rotate until the lateral deviation information is within a preset lateral deviation range.

[0028] In one embodiment, before controlling the rotation of the wheeled robot based on image information of positioning markers in images captured by the robot's camera, the method further includes:

[0029] Acquire images captured by the camera of the wheeled robot;

[0030] Control the wheeled robot to rotate until the positioning marker is included in the image captured by the camera of the wheeled robot.

[0031] Secondly, this application also provides a control device for a wheeled robot. The device includes:

[0032] The first control module is used to control the rotation of the wheeled robot based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, and to determine the first distance information between the camera and the positioning mark;

[0033] The second control module is used to control the wheeled robot to move according to the first distance information and the preset distance information between the camera and the positioning mark, and to determine the angle information between the wheeled robot and the preset reference plane according to the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0034] The third control module is used to control the rotation of the wheeled robot according to the angle information, and to determine the second distance information between the camera and the positioning mark according to the image information of the positioning mark in the image captured by the camera of the wheeled robot;

[0035] The fourth control module is used to control the wheeled robot to move to a preset position based on the second distance information.

[0036] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0037] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0038] Based on the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and the angle information between the wheeled robot and the preset reference plane is determined based on the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0039] Based on the angle information, the wheeled robot is controlled to rotate, and based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0040] Based on the second distance information, the wheeled robot is controlled to move to a preset position.

[0041] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0042] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0043] Based on the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and the angle information between the wheeled robot and the preset reference plane is determined based on the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0044] Based on the angle information, the wheeled robot is controlled to rotate, and based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0045] Based on the second distance information, the wheeled robot is controlled to move to a preset position.

[0046] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0047] Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0048] Based on the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and the angle information between the wheeled robot and the preset reference plane is determined based on the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0049] Based on the angle information, the wheeled robot is controlled to rotate, and based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0050] Based on the second distance information, the wheeled robot is controlled to move to a preset position.

[0051] The aforementioned wheeled robot control method, apparatus, computer equipment, computer-readable storage medium, and computer program product control the wheeled robot to rotate based on image information of a positioning marker in an image captured by the robot's camera, and determine a first distance between the camera and the positioning marker; control the wheeled robot to move based on the first distance information and a preset distance between the camera and the positioning marker, and determine an angle between the wheeled robot and a preset reference plane based on the image information of the positioning marker in an image captured by the robot's camera; control the wheeled robot to rotate based on the angle information, and determine a second distance between the camera and the positioning marker based on the image information of the positioning marker in an image captured by the robot's camera; and control the wheeled robot to move to a preset position based on the second distance information. This scheme divides the positioning and control process of the wheeled robot into four consecutive steps: rotational positioning, distance adjustment, angle correction, and final positioning. In each step, real-time feedback control is performed based on positioning marker image information acquired by a camera. This facilitates timely acquisition and adjustment of the robot's position and attitude information at each control stage, thereby gradually improving positioning accuracy. Through the cyclical acquisition and correction of multi-dimensional position and attitude information, this scheme achieves precise positioning control of the wheeled robot, thus improving the accuracy of wheeled robot control. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating a control method for a wheeled robot in one embodiment;

[0054] Figure 2 This is a schematic diagram of a wheeled robot in one embodiment;

[0055] Figure 3 This is another schematic diagram of the wheeled robot in one embodiment;

[0056] Figure 4 This is a schematic diagram illustrating the acquisition of images in one embodiment;

[0057] Figure 5 This is another schematic diagram illustrating the acquisition of images in one embodiment;

[0058] Figure 6 This is yet another schematic diagram of a wheeled robot in one embodiment;

[0059] Figure 7 This is a structural block diagram of a wheeled robot control device in one embodiment;

[0060] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0063] In scenarios such as automatic homing and recharging of intelligent robots, precise positioning and attitude control are required. Due to the balance control strategy of bipedal robots and the inherent slippage characteristics of wheels, uncontrollable attitude changes can occur during frequent turning and positioning. Therefore, achieving precise positioning and accurate attitude control for bipedal robots equipped with only a single camera is very difficult, usually requiring the addition of complex auxiliary sensors or expensive mechanical structures. However, various sensors may be affected by interference under different operating conditions, leading to data conflicts, complicating the software algorithm, and introducing more uncertainties. The following presents a control method for bipedal robots that achieves accurate positioning using only a monocular camera and a single marker.

[0064] In one exemplary embodiment, such as Figure 1 As shown, a control method for a wheeled robot is provided. This embodiment illustrates the application of this method to a controller (which can be a terminal) of the wheeled robot. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc.; the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. In this embodiment, the method includes the following steps:

[0065] Step S101: Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, control the wheeled robot to rotate and determine the first distance information between the camera and the positioning mark.

[0066] Step S102: Based on the first distance information and the preset distance information between the camera and the positioning mark, control the wheeled robot to move, and determine the angle information between the wheeled robot and the preset reference plane based on the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0067] Step S103: Based on the angle information, control the wheeled robot to rotate, and determine the second distance information between the camera and the positioning mark based on the image information of the positioning mark in the image captured by the wheeled robot's camera.

[0068] Step S104: Based on the second distance information, control the wheeled robot to move to the preset position.

[0069] Among them, wheeled robots can be robots with wheeled structures and balance control functions, such as biwheeled robots.

[0070] The camera can be an image acquisition device used to collect images; for example, the camera can be a monocular camera of a wheeled robot.

[0071] The location identifier can be a visual marker used for robot localization. For example, the location identifier can be an ArUco (an open-source micro augmented reality library primarily used for augmented reality applications and machine vision) marker, such as an ArUco location tag.

[0072] The images captured by the camera of the wheeled robot can be images captured by the camera at the current real-time position of the wheeled robot.

[0073] Among them, the image information of the positioning mark in the image captured by the camera of the wheeled robot can be the position and posture information of the positioning mark in the image captured at the current real-time position, such as the coordinate information and size information of the positioning mark in the image.

[0074] The first distance information can be the distance parameter between the camera and the positioning tag. For example, the first distance information can be the distance from the center of the camera to the center of the positioning tag.

[0075] The preset distance information can be a pre-set baseline distance value.

[0076] The preset reference plane can be a reference plane used to determine the robot's position. For example, the preset reference plane can be the XZ plane, where X represents the X-axis of the coordinate system and Z represents the Z-axis of the coordinate system.

[0077] The angle information can be the angle value of the wheeled robot relative to a preset reference plane. For example, the angle information can be the angle between the robot's face and the XZ plane.

[0078] The second distance information can be the distance parameter between the camera and the positioning mark after the robot completes the rotation. For example, the second distance information can be the length during the final calibration.

[0079] The preset position can be the target position that the wheeled robot needs to reach. For example, the preset position can be a position within a specified area of ​​the wheeled robot, such as the storage position (recharge position) of the wheeled robot.

[0080] Optionally, the controller of the wheeled robot acquires images from the robot's camera. The controller performs distortion correction on the acquired images, identifies ArUco positioning markers, and controls the wheeled robot's rotation based on the image information from the positioning markers. During rotation, the controller uses the Pose array provided by the ArUco positioning markers to calculate a first distance between the robot's camera and the positioning markers based on the image information from the camera images. It then controls the wheeled robot to move forward or backward based on a comparison of this first distance with a preset distance, and simultaneously calculates the angle between the wheeled robot and a preset reference plane. After acquiring the angle information, the controller controls the wheeled robot to rotate to the left until the angle between the wheeled robot and the preset reference plane is within a preset error range. Subsequently, the controller uses the Pose data provided by the ArUco positioning markers to calculate a second distance between the robot's camera and the positioning markers based on the image information from the camera images. To improve positioning accuracy, the wheeled robot's controller collects multiple distance data points, removes the maximum and minimum values, and takes the average as the final second distance information. Based on this distance information, the controller controls the wheeled robot to move forward or backward until it reaches the preset position. Throughout the process, the controller monitors distance and angle information in real time to ensure that the wheeled robot always moves within a reasonable error range.

[0081] For example, refer to Figures 2 to 6In terms of environment setup, the wheeled robot's controller needs to subscribe to ` / camera / image_raw` (sensor_msgs.msg.Image, image message) and ` / camera / camera_info` (sensor_msgs.msg.CameraInfo, camera information message). Subscribing to ` / camera / image_raw` is used to acquire camera images, while subscribing to ` / camera / camera_info` is used to acquire camera parameters. The `sensor_msgs / CameraInfo` message format is a message type used in ROS (Robot Operating System) for transmitting camera parameters, containing information such as the camera's intrinsic parameter matrix, distortion coefficients, and image width and height. By subscribing to the ` / camera / camera_info` topic, these key camera parameters can be obtained. These parameters are crucial for subsequent image processing, especially for the accurate localization and pose estimation of AR (Augmented Reality) tags. Regarding image correction and processing, camera lenses typically introduce certain distortions, such as barrel distortion or pincushion distortion. These distortions affect the shape and position of objects in the image, thus impacting the recognition accuracy of ArUco tags. By using camera parameters, especially distortion coefficients, image distortion can be corrected, resulting in more accurate image data. This is very helpful in improving the recognition rate and positioning accuracy of ArUco markers.

[0082] Regarding data correction, after generating the positioning tags, it's crucial to ensure the acquisition of their pose data. To facilitate data calculation, the center point of the positioning tag is aligned with the camera center. The robot's pose angle and the actual distance from the positioning tag's center point to the camera center are calculated and compared with the coordinates and orientation information in the 3D position space. If any deviation exists, a deviation factor needs to be calculated and used for data correction to obtain accurate pose data.

[0083] refer to Figure 2 Suppose the wheeled robot needs to automatically return to its original position within rectangle P, so that the midpoint of its head is directly above point D. Figure 2 The image shows a preset location as region P, where point D can be a point on region P, point B can be the center point of the positioning tag, point A can be a point at the same height as point B, and point C can be the center point of the camera / the center point of the wheeled robot's face.

[0084] refer to Figure 3, in terms of data calibration, the wheel-legged robot needs to automatically return to the preset rectangular area P so that the midpoint of the robot's head is vertically above point D, and the robot's face is parallel to the XZ axis of the positioning tag. Estimate the position and orientation information of the wheel-legged robot according to the Pose array provided by the ArUco positioning tag, obtain the length value of BC as CalibrationValue1 (calibration value 1), and establish a pixel coordinate system as Figure 3 (including the X-axis, Y-axis, and Z-axis), and refer to Figure 4 , and obtain the abscissa of the center point K of the positioning tag as CalibrationValue2 (calibration value 2).

[0085] Exemplarily, refer to Figures 2 to 6 :

[0086] Step 1: The wheel-legged robot continuously rotates to find its position. If it recognizes the positioning tag, it proceeds to the next step.

[0087] Step 2: Refer to Figure 5 , Figure 5 is the picture containing the positioning tag captured by the wheel-legged robot. Establish a pixel coordinate system based on the camera picture, and continue to rotate to find the position until the difference between the abscissa of the center point of the positioning tag and the abscissa of the pixel center is within the threshold range (if the total width of the picture pixels is x0, the abscissa of the center point of the tag is approximately x0 / 2). Then stop rotating and enter Step 3. Figure 5 also shows the points (0, 0), (0, y0), (x0 / 2, y0), (x0, 0), where y0 represents the total length of the picture pixels.

[0088] Step 3: Refer to Figure 6 , estimate the position and orientation of the wheel-legged robot according to the Pose array provided by the ArUco positioning tag, calculate the distance BC from the camera center to the center of the positioning tag and angle B, construct the following triangle ABE with angle A as a right angle, and find the length of BE (BE is the distance corresponding to the preset distance information) based on the fixed length AB, that is, BE = AB / COSB. If BC > BE + K1, the wheel-legged robot moves straight forward. If BC < BE - K1, the wheel-legged robot moves backward. If BE - K1 <= BC <= BE + K1, it means the wheel-legged robot is already within the error range in the YZ plane, and the wheel-legged robot stops and enters Step 4 (where K1 is the error fluctuation range value. The smaller this value is, the smaller the error, but the calibration time will increase).

[0089] Step 4: In the case of long distance, there will be errors in the position and orientation data provided by the ArUco positioning tag, and direction slippage is likely to occur during the calibration process of the wheel-legged robot. When the wheel-legged robot is stationary, take 30 sets of data for angle B and side length BC respectively, remove the maximum and minimum values and then take the average value, and compare it with the calculated value in Step 3. If they are equal within the error range, proceed to Step 5. Otherwise, go back to Step 3 and continue the iteration.

[0090] Step 5: After roughly calibrating the center of the wheel-legged robot to be in the YZ plane in the previous four steps, the following is the correction process. Control the wheel-legged robot to rotate to the left, and calculate the angle between the face of the wheel-legged robot and the XZ plane based on the Pose data provided by the ArUco positioning tag. If this angle is equal to 0 within the error range, stop rotating and then proceed to Step 6.

[0091] Step 6: Obtain the length of BC according to the position and orientation data provided by the ArUco positioning tag, and control the wheel-legged robot to move forward. If BC > CalibrationValue1 + K2, the wheel-legged robot moves straight forward. If BC < CalibrationValue1 - K2, the wheel-legged robot moves backward.

[0092] If CalibrationValue1 - K2 <= BC <= CalibrationValue1 + K2, it means that the wheel-legged robot has reached within the error range inside the P area box. The wheel-legged robot stops and proceeds to Step 7 (where K2 is the error fluctuation range value. The smaller this value is, the smaller the error, but the calibration time will increase).

[0093] Step 7: Establish a pixel coordinate system, which can refer to Figure 4 , obtain the abscissa Xp of the center point P of the tag (the center point of the currently collected tag, named point P). If |Xp - CalibrationValue2| < K3, it means that the position and orientation are correct, and successful positioning and alignment are achieved. Otherwise, the position and orientation are incorrect, and go back to Step 1 for iteration. Here, K3 is the error fluctuation range value, and this value ensures the final positioning accuracy.

[0094] In addition, in this embodiment, a single ArUco positioning tag is set on the right side, and the wheel-legged robot needs to be positioned and aligned from the left side for storage. If the wheel-legged robot needs to be able to be positioned and aligned for storage from both sides, a positioning tag can be added on the left side, and according to the same process, the pose can be corrected on both the left and right sides, and accurate positioning and storage can be achieved.

[0095] In the aforementioned wheeled robot control method, the wheeled robot is rotated based on the image information of the positioning marker in the image captured by the robot's camera, and a first distance between the camera and the positioning marker is determined. Based on the first distance and a preset distance between the camera and the positioning marker, the wheeled robot is moved, and the angle between the wheeled robot and a preset reference plane is determined based on the image information of the positioning marker in the image captured by the robot's camera. Based on the angle, the wheeled robot is rotated, and a second distance between the camera and the positioning marker is determined based on the image information of the positioning marker in the image captured by the robot's camera. Based on the second distance, the wheeled robot is moved to a preset position. This scheme divides the positioning control process of the wheeled robot into four consecutive steps: rotational positioning, distance adjustment, angle correction, and final positioning. Real-time feedback control is performed in each step based on the positioning marker image information captured by the camera. This facilitates timely acquisition and adjustment of the robot's position and attitude information at each control stage, thereby gradually improving positioning accuracy. Through the cyclical acquisition and correction of multi-dimensional position and attitude information, this scheme achieves precise positioning control of the wheeled robot, thus improving the accuracy of wheeled robot control.

[0096] In one exemplary embodiment, reference is made to Figure 6 Based on the first distance information and the preset distance information between the camera and the positioning marker, the wheeled robot is controlled to move. Specifically, this includes: determining the angle information formed by the center point of the camera and the center point of the positioning marker on a preset reference plane based on the image information of the positioning marker in the image captured by the wheeled robot's camera; determining the preset distance information based on the angle information and the lateral distance information between the preset position and the center point of the positioning marker; and controlling the wheeled robot to move based on the first distance information until the first distance information is within the preset distance range corresponding to the preset distance information.

[0097] The center point of the positioning marker can be a point marked by ArUco in the image, for example, the center point of the positioning marker can be the marker center point K.

[0098] The included angle information can be the angle formed by the center point of the camera and the center point of the positioning mark on a preset reference plane. For example, the included angle information can be the angle of angle B in triangle ABE.

[0099] The lateral distance information can be the horizontal distance from the preset location to the center point of the positioning marker. For example, the lateral distance information can be a fixed length AB.

[0100] The preset distance range can be the allowable distance error range. For example, when the preset distance information is BE, the preset distance range can be the range from BE-K1 to BE+K1 (where K1 is the error fluctuation range value).

[0101] Optionally, the controller of the wheeled robot uses the Pose array provided by the ArUco positioning marker to calculate the angle information, i.e., angle B, formed by the center point of the camera and the center point of the positioning marker on the preset reference plane YZ plane, based on the image information of the positioning marker in the images captured by the wheeled robot's camera (for example, by matching the image information of the positioning marker in the images captured by the wheeled robot's camera with the image information corresponding to the Pose array). Using this angle information and a fixed length AB as known conditions, a right triangle ABE is constructed, and the preset distance information BE is calculated. Subsequently, the controller of the wheeled robot determines whether to move forward or backward based on the difference between the first distance information BC and the preset distance information BE, until the first distance information BC is within the preset distance range corresponding to the preset distance information BE.

[0102] The technical solution provided in this embodiment determines the preset distance information based on the angle information formed by the center point of the camera and the center point of the positioning mark, as well as the lateral distance information between the preset position and the center point of the positioning mark. This helps to establish accurate spatial geometric relationships, thereby improving the positioning accuracy of the wheeled robot. By adjusting the position by controlling the first distance information within the preset distance range, positioning errors can be effectively reduced, ensuring that the wheeled robot can accurately reach the preset position.

[0103] In an exemplary embodiment, determining the first distance information between the camera and the positioning marker specifically includes the following: determining basic first distance information between the camera and the positioning marker based on the image information of the positioning marker in the image captured by the wheeled robot's camera; if the basic first distance information is greater than a preset distance threshold, determining multiple candidate first distance information between the camera and the positioning marker based on the image information of the positioning marker in the image captured by the wheeled robot's camera; determining the first distance information based on the average information among the multiple candidate first distance information; determining the angle information formed by the center point of the camera and the center point of the positioning marker on a preset reference plane based on the image information of the positioning marker in the image captured by the wheeled robot's camera, specifically including the following: determining multiple candidate angle information formed by the center point of the camera and the center point of the positioning marker on a preset reference plane based on the image information of the positioning marker in the image captured by the wheeled robot's camera; determining the angle information based on the average information among the multiple candidate angle information.

[0104] Among them, reference Figure 2 and Figure 3 The basic first distance information can be the initial distance data between the camera and the positioning identifier. For example, the basic first distance information can be the BC length estimated for the first time based on the Pose array provided by the ArUco positioning identifier.

[0105] The preset distance threshold can be a distance reference value used to determine whether multiple measurements are needed. For example, the preset distance threshold can be the distance limit at which the position and attitude data provided by the ArUco positioning marker may produce errors in the case of long distances.

[0106] Among them, reference Figure 6 The candidate first distance information can be a single distance data in a set of distance data obtained from multiple measurements. For example, the candidate first distance information can be 30 sets of BC side length data collected when the robot is stationary.

[0107] The average information can be the average value obtained after statistical processing of multiple data. For example, the average information can be the result of averaging the remaining data after removing the maximum and minimum values.

[0108] Among them, reference Figure 6 Candidate angle information can be a single angle data from a set of angle data obtained from multiple measurements. For example, candidate angle information can be 30 sets of angle B data collected when the robot is stationary.

[0109] Optionally, refer to Figure 6 The controller of the wheeled robot estimates the position and posture based on the Pose array provided by the ArUco positioning marker and calculates the basic first distance information BC between the camera and the positioning marker. When the basic first distance information BC is greater than a preset distance threshold, the controller keeps the wheeled robot stationary and continuously collects 30 sets of candidate first distance information BC between the camera and the positioning marker. Simultaneously, the controller collects the candidate angle information B formed by the center points of the 30 cameras and the center point of the positioning marker on the preset reference plane YZ plane. After acquiring the 30 sets of candidate first distance information BC, the controller removes the maximum and minimum values ​​and performs an arithmetic average on the remaining 28 sets of candidate first distance information BC to obtain the final first distance information BC. Similarly, the controller also removes the maximum and minimum values ​​from the 30 sets of candidate angle information B and performs an arithmetic average on the remaining 28 sets of candidate angle information B to obtain the final angle information B. This method can effectively reduce the error in position and attitude data provided by ArUco positioning markers in long-distance situations, as well as the error caused by directional slippage that may occur during the calibration process of wheeled robots.

[0110] The technical solution provided in this embodiment collects multiple candidate first distance information and multiple candidate included angle information when the basic first distance information is greater than a preset distance threshold, and averages these information. This helps to eliminate the single measurement error caused by the reduction of the ArUco code projection size under long-distance conditions, thereby improving the accuracy of wheeled robots in long-distance positioning.

[0111] In an exemplary embodiment, after controlling the wheeled robot to move to a preset position based on the second distance information, the method further includes: obtaining the current horizontal coordinate of the center point of the positioning marker in the image captured by the wheeled robot's camera; and determining whether the wheeled robot has successfully moved to the preset position based on the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning marker.

[0112] The current horizontal coordinate can be the real-time horizontal position parameter of the center point of the label in the pixel coordinate system. For example, the current horizontal coordinate can be the horizontal coordinate Xp of the label center point P.

[0113] The preset horizontal coordinate can be the standard horizontal position parameter of the center point of the positioning mark obtained during the data calibration stage. For example, the preset horizontal coordinate can be the horizontal coordinate CalibrationValue2 of the label center point K obtained when the robot is inside the rectangle and the center point of the robot's face is vertically above point D.

[0114] Optionally, the controller of the wheeled robot establishes a pixel coordinate system in the image, obtains the current horizontal coordinate Xp of the center point P of the positioning marker through the Pose array provided by the ArUco positioning marker, and compares the current horizontal coordinate Xp with the preset horizontal coordinate CalibrationValue2 to determine whether the wheeled robot has successfully moved to the preset position.

[0115] The technical solution provided in this embodiment, by obtaining the current horizontal coordinate of the center point of the positioning marker and comparing it with the preset horizontal coordinate, is beneficial to accurately verify the actual position of the wheeled robot after the movement is completed. This helps to solve the positioning deviation problem caused by slippage that may occur during the movement of the wheeled robot, and ensures that the wheeled robot can accurately reach the preset position.

[0116] In an exemplary embodiment, controlling the rotation of the wheeled robot based on the image information of the positioning marker in the image captured by the camera of the wheeled robot specifically includes the following: performing image recognition processing on the image captured by the camera of the wheeled robot to obtain the image information of the positioning marker in the image captured by the camera of the wheeled robot; determining the lateral deviation information between the center point of the positioning marker and the center point of the image captured by the camera of the wheeled robot based on the image information of the positioning marker in the image captured by the camera of the wheeled robot; and controlling the rotation of the wheeled robot based on the lateral deviation information until the lateral deviation information is within a preset lateral deviation range.

[0117] Image recognition processing can be the process of correcting distortion and recognizing features in images captured by a camera, and identifying ArUco location markers.

[0118] The lateral deviation information can be the difference in horizontal distance between the center point of the positioning marker and the center point of the image captured by the camera.

[0119] The preset lateral deviation range can be the maximum allowable range of lateral deviations.

[0120] The center point of the captured image can be the center position of the image captured by the camera in the pixel coordinate system. For example, the center point of the captured image can be the pixel position corresponding to the total width of the image pixels x0 / 2.

[0121] Optionally, the controller of the wheeled robot performs ArUco positioning tag recognition processing on the corrected image. A pixel coordinate system is established in the image, and half the total pixel width x0 of the image is calculated as the abscissa of the center point of the image captured by the camera. Simultaneously, the abscissa of the center point of the positioning tag is acquired. The difference between these two abscissas is calculated to obtain the lateral deviation information. After acquiring the lateral deviation information, the controller of the wheeled robot determines whether the deviation is within a preset lateral deviation range. If the lateral deviation is positive and exceeds the preset range, the controller controls the wheeled robot to rotate to the right; if the lateral deviation is negative and exceeds the preset range, the controller controls the wheeled robot to rotate to the left; when the lateral deviation enters the preset range, the controller stops the rotation of the wheeled robot. This rotation control method based on lateral deviation ensures that the positioning tag remains in the center of the camera's field of view, thereby achieving the best recognition effect.

[0122] The technical solution provided in this embodiment calculates the lateral deviation information between the center point of the positioning marker and the center point of the image acquired by the camera, and controls the rotation of the wheeled robot based on the deviation information. This helps to keep the positioning marker always in the center of the camera's field of view, thereby facilitating the best image recognition effect. The lateral deviation control method achieves dynamic alignment between the center of the camera and the center of the positioning marker, ensuring the accuracy of the subsequent positioning process.

[0123] In an exemplary embodiment, before controlling the rotation of the wheeled robot based on the image information of the positioning marker in the image captured by the camera of the wheeled robot, the following steps are also included: acquiring the image captured by the camera of the wheeled robot; controlling the rotation of the wheeled robot until the image captured by the camera of the wheeled robot contains the positioning marker.

[0124] Optionally, the controller of the wheeled robot acquires images and camera parameter information from the robot's camera. The controller uses the camera parameters to perform distortion correction on the acquired images to eliminate barrel or pincushion distortion caused by the camera lens. The controller controls the wheeled robot to rotate at a fixed angular velocity. During rotation, the controller continuously performs ArUco positioning mark recognition processing on the images acquired by the robot's camera. When the controller successfully recognizes a complete ArUco positioning mark in the image acquired by the robot's camera, it stops the robot's rotation.

[0125] The technical solution provided in this embodiment, by controlling the rotation of the wheeled robot until the image captured by its camera contains the positioning marker, helps to ensure that the positioning marker has entered the field of view of the camera before accurate positioning control is performed. This helps to avoid invalid image recognition and position calculation when the positioning marker is not in the field of view, thereby improving the efficiency of the entire positioning process.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0127] Based on the same inventive concept, this application also provides a wheeled robot control device for implementing the wheeled robot control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more wheeled robot control device embodiments provided below can be found in the limitations of the wheeled robot control method described above, and will not be repeated here.

[0128] In one exemplary embodiment, such as Figure 7 As shown, a wheeled robot control device 700 is provided, which may include:

[0129] The first control module 701 is used to control the rotation of the wheeled robot based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, and to determine the first distance information between the camera and the positioning mark.

[0130] The second control module 702 is used to control the wheeled robot to move according to the first distance information and the preset distance information between the camera and the positioning mark, and to determine the angle information between the wheeled robot and the preset reference plane according to the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0131] The third control module 703 is used to control the rotation of the wheeled robot according to the angle information, and to determine the second distance information between the camera and the positioning mark according to the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0132] The fourth control module 704 is used to control the wheeled robot to move to a preset position based on the second distance information.

[0133] In an exemplary embodiment, the second control module 702 is further configured to determine the angle information formed by the center point of the camera and the center point of the positioning mark on a preset reference plane based on the image information of the positioning mark in the image captured by the camera of the wheeled robot; determine the preset distance information based on the angle information and the lateral distance information between the preset position and the center point of the positioning mark; and control the wheeled robot to move according to the first distance information until the first distance information is within the preset distance range corresponding to the preset distance information.

[0134] In an exemplary embodiment, the first control module 701 is further configured to determine basic first distance information between the camera and the positioning marker based on the image information of the positioning marker in the image captured by the camera of the wheeled robot; if the basic first distance information is greater than a preset distance threshold, determine multiple candidate first distance information between the camera and the positioning marker based on the image information of the positioning marker in the image captured by the camera of the wheeled robot; and determine the first distance information based on the average information between the multiple candidate first distance information. The second control module 702 is further configured to determine multiple candidate angle information formed by the center point of the camera and the center point of the positioning marker on a preset reference plane based on the image information of the positioning marker in the image captured by the camera of the wheeled robot; and determine the angle information based on the average information between the multiple candidate angle information.

[0135] In an exemplary embodiment, the device 700 further includes: a coordinate acquisition module, configured to acquire the current abscissa of the center point of the positioning marker in the image captured by the camera of the wheeled robot; and determine whether the wheeled robot has successfully moved to the preset position based on the current abscissa and the preset abscissa of the center point of the positioning marker.

[0136] In an exemplary embodiment, the first control module 701 is further configured to perform image recognition processing on the images captured by the camera of the wheeled robot to obtain image information of the positioning mark in the images captured by the camera of the wheeled robot; determine the lateral deviation information between the center point of the positioning mark and the center point of the images captured by the camera of the wheeled robot based on the image information of the positioning mark in the images captured by the camera of the wheeled robot; and control the wheeled robot to rotate based on the lateral deviation information until the lateral deviation information is within a preset lateral deviation range.

[0137] In an exemplary embodiment, the device 700 further includes: an image acquisition module for acquiring images captured by the camera of the wheeled robot; and controlling the wheeled robot to rotate until the images captured by the camera of the wheeled robot contain a positioning marker.

[0138] Each module in the aforementioned wheeled robot control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0139] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a wheeled robot control method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0140] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0141] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0142] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0143] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0144] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0145] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A control method for a wheeled legged robot, characterized in that, The method includes: Based on the image information of the positioning marker in the image captured by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and based on the image information of the positioning marker in the image captured by the camera of the wheeled robot, the basic first distance information between the camera and the positioning marker is determined; If the basic first distance information is greater than a preset distance threshold, multiple candidate first distance information between the camera and the positioning marker are determined based on the image information of the positioning marker in the image captured by the camera of the wheeled robot. The first distance information between the camera and the positioning identifier is determined based on the average information among the multiple candidate first distance information. Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, determine multiple candidate angle information formed by the center point of the camera and the center point of the positioning mark on a preset reference plane; Based on the average information among the multiple candidate angle information, the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane is determined; Based on the included angle information and the lateral distance information between the preset position and the center point of the positioning mark, the preset distance information between the camera and the positioning mark is determined; Based on the first distance information, the wheeled robot is controlled to move until the first distance information is within the preset distance range corresponding to the preset distance information. Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the angle information between the wheeled robot and the preset reference plane is determined. Based on the angle information, the wheeled robot is controlled to rotate, and based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined; Based on the second distance information, the wheeled robot is controlled to move to the preset position.

2. The method according to claim 1, characterized in that, The wheeled robot is a biwheeled robot.

3. The method according to claim 2, characterized in that, The camera is a monocular camera of the wheeled robot.

4. The method according to claim 1, characterized in that, After controlling the wheeled robot to move to a preset position based on the second distance information, the method further includes: Obtain the current abscissa of the center point of the positioning marker in the image captured by the camera of the wheeled robot; Based on the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning mark, determine whether the wheeled robot has successfully moved to the preset position.

5. The method according to claim 1, characterized in that, The step of controlling the rotation of the wheeled robot based on the image information of the positioning markers in the images captured by the camera of the wheeled robot includes: Image recognition processing is performed on the images captured by the camera of the wheeled robot to obtain the image information of the positioning marker in the images captured by the camera of the wheeled robot; Based on the image information of the positioning mark in the image captured by the camera of the wheeled robot, the lateral deviation information between the center point of the positioning mark and the center point of the image captured by the camera of the wheeled robot is determined. Based on the lateral deviation information, the wheeled robot is controlled to rotate until the lateral deviation information is within a preset lateral deviation range.

6. The method according to any one of claims 1 to 5, characterized in that, Before controlling the rotation of the wheeled robot based on the image information of the positioning markers in the images captured by the camera of the wheeled robot, the process also includes: Acquire images captured by the camera of the wheeled robot; Control the wheeled robot to rotate until the positioning marker is included in the image captured by the camera of the wheeled robot.

7. A control device for a wheeled robot, which executes the control method for a wheeled robot as described in claim 1, characterized in that, The device includes: The first control module is configured to control the rotation of the wheeled robot based on the image information of the positioning marker in the image captured by the wheeled robot's camera, and determine a basic first distance information between the camera and the positioning marker based on the image information of the positioning marker in the image captured by the wheeled robot's camera; if the basic first distance information is greater than a preset distance threshold, determine multiple candidate first distance information between the camera and the positioning marker based on the image information of the positioning marker in the image captured by the wheeled robot's camera; and determine the first distance information between the camera and the positioning marker based on the average information among the multiple candidate first distance information. The second control module is configured to: determine multiple candidate angles formed by the center point of the camera and the center point of the positioning mark on a preset reference plane based on image information of the positioning mark in the images captured by the camera of the wheeled robot; determine the angle between the center point of the camera and the center point of the positioning mark on the preset reference plane based on the average information among the multiple candidate angles; determine a preset distance between the camera and the positioning mark based on the angle information and the lateral distance information between a preset position and the center point of the positioning mark; control the wheeled robot to move according to the first distance information until the first distance information is within the preset distance range corresponding to the preset distance information; and determine the angle information between the wheeled robot and the preset reference plane based on the image information of the positioning mark in the images captured by the camera of the wheeled robot. The third control module is used to control the rotation of the wheeled robot according to the angle information, and to determine the second distance information between the camera and the positioning mark according to the image information of the positioning mark in the image captured by the camera of the wheeled robot; The fourth control module is used to control the wheeled robot to move to the preset position based on the second distance information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images