Wheel-foot robot control method, device and equipment and readable storage medium

Abstract

The invention relates to a wheel-foot robot control method and device, computer equipment, a computer readable storage medium and a computer program product, and can be applied to the technical field of robots. The method comprises the following steps: controlling the wheel-foot robot to rotate according to image information of a positioning identifier in an image collected by a camera of the wheel-foot robot, and determining first distance information between the camera and the positioning identifier; controlling the wheel-foot robot to move according to the first distance information and preset distance information between the camera and the positioning identifier, and determining angle information between the wheel-foot robot and a preset reference plane according to image information of the positioning identifier in the acquired image; controlling the wheel-foot robot to rotate according to the angle information, and determining second distance information between the camera and the positioning identifier according to the image information of the positioning identifier; and controlling the wheel-foot robot to move to a preset position according to the second distance information. By adopting the method, the control accuracy of the wheel-foot robot can be improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a wheeled robot control method, device, equipment and computer-readable storage medium. Background Art

[0002] With the development of robotics technology, the demand for positioning control and attitude control of wheeled robots in application scenarios such as intelligent automatic homing and automatic recharging has become increasingly prominent. Due to its own balance control strategy and wheel slip characteristics, wheeled robots are prone to uncontrollable attitude changes during frequent steering and positioning. Therefore, how to achieve positioning control and attitude control of wheeled robots has become an important research direction.

[0003] Traditional technology usually adds multiple auxiliary sensors or complex structures to perform positioning and posture control of wheeled robots. However, various sensors are easily interfered with in different working conditions, resulting in data differences and data conflicts, which not only increases the complexity of software algorithms, but also brings more uncertainties. In particular, for wheeled robots with only a single camera, the control accuracy of the wheeled robot is low. Summary of the invention

[0004] Based on this, it is necessary to provide a wheeled-legged robot control method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of wheeled-legged robot control in order to solve the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for controlling a wheeled robot. The method comprises:

[0006] According to the image information of the positioning mark in the image collected by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0007] According to the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the angle information between the wheeled robot and the preset reference plane is determined;

[0008] According to the angle information, the wheeled robot is controlled to rotate, and according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0009] According to the second distance information, the wheeled robot is controlled to move to a preset position.

[0010] In one embodiment, controlling the movement of the wheeled robot according to the first distance information and the preset distance information between the camera and the positioning mark includes:

[0011] According to the image information of the positioning mark in the captured image of the camera of the wheeled robot, determine the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane;

[0012] Determining the preset distance information according to the angle information and the lateral distance information between the preset position and the center point of the positioning mark;

[0013] According to the first distance information, the wheeled robot is controlled to move until the first distance information is within a preset distance range corresponding to the preset distance information.

[0014] In one embodiment, determining the first distance information between the camera and the positioning mark includes:

[0015] Determine basic first distance information between the camera and the positioning mark according to image information of the positioning mark in the image collected by the camera of the wheeled robot;

[0016] When the basic first distance information is greater than a preset distance threshold, a plurality of candidate first distance information between the camera and the positioning mark is determined according to the image information of the positioning mark in the captured image of the camera of the wheeled robot;

[0017] Determining the first distance information according to average information among the plurality of candidate first distance information;

[0018] The method of determining the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane based on the image information of the positioning mark in the captured image of the camera of the wheeled robot includes:

[0019] According to the image information of the positioning mark in the captured image of the camera of the wheeled robot, determine a plurality of candidate angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane;

[0020] The angle information is determined according to average information among the plurality of candidate angle information.

[0021] In one embodiment, after controlling the wheeled robot to move to a preset position according to the second distance information, the method further includes:

[0022] Obtaining the current horizontal coordinate of the center point of the positioning mark in the captured image of the camera of the wheeled robot;

[0023] According to the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning mark, it is determined whether the wheeled robot has successfully moved to the preset position.

[0024] In one embodiment, controlling the wheeled robot to rotate according to image information of a positioning mark in an image captured by a camera of the wheeled robot comprises:

[0025] Performing image recognition processing on the image collected by the camera of the wheeled robot to obtain image information of the positioning mark in the image collected by the camera of the wheeled robot;

[0026] Determine, according to image information of the positioning mark in the image captured by the camera of the wheeled robot, lateral deviation information between the center point of the positioning mark and the center point of the image captured by the camera of the wheeled robot;

[0027] According to the lateral deviation information, the wheeled robot is controlled to rotate until the lateral deviation information is within a preset lateral deviation range.

[0028] In one embodiment, before controlling the wheeled robot to rotate according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, the method further includes:

[0029] Acquire an image captured by a camera of the wheeled robot;

[0030] The wheeled robot is controlled to rotate until the image captured by the camera of the wheeled robot contains the positioning mark.

[0031] In a second aspect, the present application also provides a wheeled robot control device. The device comprises:

[0032] A first control module is used to control the wheeled robot to rotate according to image information of the positioning mark in the image collected by the camera of the wheeled robot, and determine first distance information between the camera and the positioning mark;

[0033] a second control module, configured to control the movement of the wheeled robot according to the first distance information and the preset distance information between the camera and the positioning mark, and determine the angle information between the wheeled robot and a preset reference plane according to the image information of the positioning mark in the image captured by the camera of the wheeled robot;

[0034] a third control module, configured to control the wheeled robot to rotate according to the angle information, and determine second distance information between the camera and the positioning mark according to image information of the positioning mark in an image captured by a camera of the wheeled robot;

[0035] The fourth control module is used to control the wheeled robot to move to a preset position according to the second distance information.

[0036] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0037] According to the image information of the positioning mark in the image collected by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0038] According to the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the angle information between the wheeled robot and the preset reference plane is determined;

[0039] According to the angle information, the wheeled robot is controlled to rotate, and according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0040] According to the second distance information, the wheeled robot is controlled to move to a preset position.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0042] According to the image information of the positioning mark in the image collected by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0043] According to the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the angle information between the wheeled robot and the preset reference plane is determined;

[0044] According to the angle information, the wheeled robot is controlled to rotate, and according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0045] According to the second distance information, the wheeled robot is controlled to move to a preset position.

[0046] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0047] According to the image information of the positioning mark in the image collected by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined;

[0048] According to the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the angle information between the wheeled robot and the preset reference plane is determined;

[0049] According to the angle information, the wheeled robot is controlled to rotate, and according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined;

[0050] According to the second distance information, the wheeled robot is controlled to move to a preset position.

[0051] The above-mentioned wheeled robot control method, device, computer equipment, computer-readable storage medium and computer program product control the wheeled robot to rotate according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, and determine the first distance information between the camera and the positioning mark; control the wheeled robot to move according to the first distance information and the preset distance information between the camera and the positioning mark, and determine the angle information between the wheeled robot and a preset reference plane according to the image information of the positioning mark in the image captured by the camera of the wheeled robot; control the wheeled robot to rotate according to the angle information, and determine the second distance information between the camera and the positioning mark according to the image information of the positioning mark in the image captured by the camera of the wheeled robot; and control the wheeled robot to move to a preset position according to the second distance information. This scheme divides the positioning control process of the wheeled robot into four consecutive steps: rotation positioning, distance adjustment, angle correction and final positioning, and performs real-time feedback control in each step based on the positioning identification image information collected by the camera. This is beneficial for timely acquisition and adjustment of the robot's position and posture information in each control stage, thereby gradually improving the positioning accuracy. This scheme realizes precise positioning control of the wheeled robot through the cyclic acquisition and correction of multi-dimensional position and posture information, thereby improving the accuracy of wheeled robot control. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A schematic diagram of a process flow of a wheeled robot control method in one embodiment;

[0054] Figure 2 is a schematic diagram of a wheeled robot in one embodiment;

[0055] Figure 3 Another schematic diagram of a wheeled robot in one embodiment;

[0056] Figure 4 A schematic diagram of capturing an image in an embodiment;

[0057] Figure 5 Another schematic diagram of capturing an image in an embodiment;

[0058] Figure 6 is another image schematic diagram of a wheeled robot in one embodiment;

[0059] Figure 7 is a structural block diagram of a wheeled-foot robot control device in one embodiment;

[0060] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0063] In scenarios such as automatic homing and automatic recharging of intelligent robots, relatively precise positioning control and attitude control are required. Due to the balance control strategy of the two-wheeled robot and the slipping characteristics of the wheels themselves, uncontrollable attitude changes will occur during frequent steering and positioning. Therefore, it is very difficult to achieve precise positioning and accurate attitude control for a wheeled robot with only a single camera. It usually requires the addition of complex auxiliary sensors or expensive mechanical structures. However, various sensors may be interfered with and produce differences in different working conditions, which in turn leads to data conflicts, complicating the software algorithm and generating more uncertainty. The following provides a control method for a wheeled robot, which can achieve accurate positioning of the wheeled robot using only a monocular camera and single marker positioning.

[0064] In an exemplary embodiment, Figure 1 As shown, a control method for a wheeled robot is provided. This embodiment uses the method applied to a controller (which may be a terminal) of a wheeled robot as an example for illustration. It is understandable that the method may also be applied to a server, or to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc.; the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. In this embodiment, the method includes the following steps:

[0065] Step S101, according to the image information of the positioning mark in the image collected by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined.

[0066] Step S102, controlling the movement of the wheeled robot according to the first distance information and the preset distance information between the camera and the positioning mark, and determining the angle information between the wheeled robot and the preset reference plane according to the image information of the positioning mark in the captured image of the camera of the wheeled robot.

[0067] Step S103, controlling the wheeled robot to rotate according to the angle information, and determining the second distance information between the camera and the positioning mark according to the image information of the positioning mark in the image captured by the camera of the wheeled robot.

[0068] Step S104, controlling the wheeled robot to move to a preset position according to the second distance information.

[0069] Among them, the wheel-legged robot can be a robot with a wheeled structure and a balance control function, such as a two-wheeled robot.

[0070] The camera may be an image acquisition device for acquiring images, for example, the camera may be a monocular camera of a wheeled robot.

[0071] The positioning mark may be a visual mark used for robot positioning, for example, the positioning mark may be an ArUco (an open source micro-reality augmented library, mainly used for augmented reality applications and machine vision) mark, such as an ArUco positioning tag.

[0072] The image captured by the camera of the wheeled-legged robot may be an image captured by the camera of the wheeled-legged robot at the current real-time position.

[0073] Among them, the image information of the positioning mark in the image captured by the camera of the wheeled robot can be the position information and posture information of the positioning mark in the image captured at the current real-time position, for example, including the coordinate information and size information of the positioning mark in the image.

[0074] The first distance information may be a distance parameter between the camera and the positioning tag. For example, the first distance information may be a distance from the center of the camera to the center of the positioning tag.

[0075] The preset distance information may be a preset reference distance value.

[0076] The preset reference plane may be a reference plane for determining the position of the robot. For example, the preset reference plane may be an XZ plane, where X represents the X axis of the coordinate axis and Z represents the Z axis of the coordinate axis.

[0077] The angle information may be an angle value of the wheeled robot relative to a preset reference plane, for example, the angle information may be an angle between the face of the robot and the XZ plane.

[0078] The second distance information may be a distance parameter between the camera and the positioning mark after the robot completes the rotation. For example, the second distance information may be the length at the final calibration.

[0079] Among them, the preset position can be a target position that the wheeled robot needs to reach. For example, the preset position can be a position within a specified area of ​​the wheeled robot, such as a storage location (recharging position) of the wheeled robot.

[0080] Optionally, the controller of the wheel-foot robot obtains the captured image of the camera of the wheel-foot robot. The controller of the wheel-foot robot performs distortion correction processing on the captured image, identifies the ArUco positioning mark, and controls the wheel-foot robot to rotate according to the image information of the positioning mark. During the rotation process, the controller of the wheel-foot robot calculates the first distance information between the camera of the wheel-foot robot and the positioning mark according to the image information of the positioning mark in the captured image of the camera of the wheel-foot robot through the Pose (pose / posture) array provided by the ArUco positioning mark, and controls the wheel-foot robot to move forward or backward according to the comparison result of the first distance information and the preset distance information, and calculates the angle information between the wheel-foot robot and the preset reference plane at the same time. After obtaining the angle information, the controller of the wheel-foot robot controls the wheel-foot robot to rotate to the left until the angle between the wheel-foot robot and the preset reference plane is within the preset error range. Subsequently, the controller of the wheel-foot robot calculates the second distance information between the camera of the wheel-foot robot and the positioning mark according to the image information of the positioning mark in the captured image of the camera of the wheel-foot robot through the Pose data provided by the ArUco positioning mark. To improve positioning accuracy, the controller of the wheeled robot collects multiple distance data and removes the maximum and minimum values ​​and takes the average value as the final second distance information. According to the distance information, the wheeled robot is controlled to move forward or backward until it reaches the preset position. During the whole process, the controller of the wheeled robot monitors the distance and angle information in real time to ensure that the wheeled robot always moves within a reasonable error range.

[0081] For example, refer to Figures 2 to 6In terms of environment construction, the controller of the wheeled robot needs to subscribe to / camera / image_raw (sensor_msgs.msg.Image, image message) and / camera / camera_info (sensor_msgs.msg.CameraInfo, camera information message), where subscribing to / camera / image_raw is used to obtain camera images, and subscribing to / camera / camera_info is used to obtain camera parameters. The sensor_msgs / CameraInfo (camera information) message format is a message type used to transmit camera parameters in ROS (Robot Operating System), which contains the camera's intrinsic parameter matrix, distortion coefficient, image width and height, and other information. By subscribing to the / camera / camera_info topic, these key parameters of the camera can be obtained, which are crucial for subsequent image processing, especially the precise positioning and pose estimation of AR (augmented reality) markers. In terms of image correction and processing, the camera lens usually introduces certain distortions, such as barrel distortion or pincushion distortion, which affect the shape and position of objects in the image, thereby affecting the recognition accuracy of ArUco markers. Through the camera parameters, especially the distortion coefficient, the image distortion can be corrected to obtain more accurate image data, which is very helpful for improving the recognition rate and positioning accuracy of ArUco markers.

[0082] In terms of data correction, after generating the positioning tag, it is necessary to ensure that the posture data of the positioning tag is obtained. In order to facilitate data calculation, the center point of the positioning tag is at the same height as the center of the camera, and the posture angle of the wheeled robot and the actual distance from the center point of the positioning tag to the center of the camera are calculated, and compared with the coordinates and orientation information in the position three-dimensional space. If there is a deviation, it is necessary to calculate the deviation factor and use the deviation factor to perform data correction to obtain accurate posture data.

[0083] refer to Figure 2 , assuming that the wheeled robot needs to automatically return to the P rectangular frame so that the midpoint of the wheeled robot's head is just above point D. Figure 2 In the figure, the preset position is shown as area P, point D can be a point on area P, point B can be the center point of the positioning tag, point A can be a point at the same height as point B, and point C can be the center point of the camera / midpoint of the face of the wheeled robot.

[0084] refer to Figure 3, in terms of data calibration, the wheel-legged robot needs to automatically return to the preset rectangular area P so that the midpoint of the robot's head is vertically above point D, and the robot's face is parallel to the XZ axis of the positioning tag. Estimate the position and pose information of the wheel-legged robot according to the Pose array provided by the ArUco positioning tag, obtain the length value of BC as CalibrationValue1 (calibration value 1), and establish a pixel coordinate system as Figure 3 (including the X-axis, Y-axis, and Z-axis), and refer to Figure 4 , and obtain the abscissa of the center point K of the positioning tag as CalibrationValue2 (calibration value 2).

[0085] Exemplarily, refer to Figures 2 to 6 :

[0086] Step 1: The wheel-legged robot continuously rotates to search for the position. If the positioning tag is recognized, proceed to the next step.

[0087] Step 2: Refer to Figure 5 , Figure 5 is the picture containing the positioning tag captured by the wheel-legged robot. Establish a pixel coordinate system based on the camera picture, and continue to rotate to search for the position until the difference between the abscissa of the center point of the positioning tag and the abscissa of the pixel center is within the threshold range (if the total width of the picture pixels is x0, the abscissa of the center point of the tag is approximately x0 / 2), then stop rotating and enter Step 3. Figure 5 also shows the points (0, 0), (0, y0), (x0 / 2, y0), (x0, 0), where y0 represents the total length of the picture pixels.

[0088] Step 3: Refer to Figure 6 , estimate the position and pose of the wheel-legged robot according to the Pose array provided by the ArUco positioning tag, and calculate the distance BC and angle B from the camera center to the center of the positioning tag. Construct the following triangle ABE with angle A as a right angle, and find the length of BE (BE is the distance corresponding to the preset distance information) according to the fixed length AB, that is, BE = AB / COSB. If BC > BE + K1, the wheel-legged robot goes straight forward. If BC < BE - K1, the wheel-legged robot moves backward. If BE - K1 <= BC <= BE + K1, it means that the wheel-legged robot is already in the YZ plane within the error range, and the wheel-legged robot stops and enters Step 4 (where K1 is the error fluctuation range value. The smaller this value is, the smaller the error, but the calibration time will increase).

[0089] Step 4: In the case of long distance, there will be errors in the position and attitude data provided by the ArUco positioning tag, and the calibration process of the wheel-legged robot is likely to cause directional slippage. Then, when the wheel-legged robot is stationary, take 30 sets of data for each of angle B and the length of BC side, remove the maximum and minimum values and then take the average, and compare it with the calculated value in Step 3. If they are equal within the error range, proceed to Step 5. Otherwise, go back to Step 3 and continue the iteration.

[0090] Step 5: After calibrating the wheel-legged robot in the previous four steps, its center is roughly in the YZ plane. Next is the correction process. Control the wheel-legged robot to rotate left, and calculate the angle between the face of the wheel-legged robot and the XZ plane according to the Pose data provided by the ArUco positioning tag. If this angle is equal to 0 within the error range, stop rotating and then proceed to Step 6.

[0091] Step 6: Obtain the length of BC according to the position and attitude data provided by the ArUco positioning tag, and control the wheel-legged robot to move forward. If BC > CalibrationValue1 + K2, the wheel-legged robot goes straight forward. If BC < CalibrationValue1 - K2, the wheel-legged robot moves backward.

[0092] If CalibrationValue1 - K2 <= BC <= CalibrationValue1 + K2, it means that the wheel-legged robot has reached within the error range inside the P area box. The wheel-legged robot stops and proceeds to Step 7 (where K2 is the error fluctuation range value. The smaller this value, the smaller the error, but the calibration time will increase).

[0093] Step 7: Establish a pixel coordinate system, which can refer to Figure 4 , obtain the abscissa Xp of the center point P of the tag (the currently collected center point of the tag, named point P). If |Xp - CalibrationValue2| < K3, it means that the position and attitude are correct, and the positioning and alignment are successful. Otherwise, the position and attitude are incorrect, and go back to Step 1 for iteration. Among them, K3 is the error fluctuation range value, and this value ensures the final positioning accuracy.

[0094] In addition, in this embodiment, a single ArUco positioning tag is set on the right side, and the wheel-legged robot needs to be positioned and aligned from the left side for storage. If the wheel-legged robot needs to be positioned and aligned for storage from both sides, a positioning tag can be added on the left side, and according to the same process, the pose can be corrected on both the left and right sides, and the storage can be accurately positioned.

[0095] In the above-mentioned wheel-foot robot control method, according to the image information of the positioning mark in the captured image of the camera of the wheel-foot robot, the wheel-foot robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined; according to the first distance information and the preset distance information between the camera and the positioning mark, the wheel-foot robot is controlled to move, and according to the image information of the positioning mark in the captured image of the camera of the wheel-foot robot, the angle information between the wheel-foot robot and the preset reference plane is determined; according to the angle information, the wheel-foot robot is controlled to rotate, and according to the image information of the positioning mark in the captured image of the camera of the wheel-foot robot, the second distance information between the camera and the positioning mark is determined; according to the second distance information, the wheel-foot robot is controlled to move to a preset position. This scheme divides the positioning control process of the wheel-foot robot into four consecutive steps of rotation positioning, distance adjustment, angle correction and final positioning, and performs real-time feedback control based on the positioning mark image information captured by the camera in each step, which is conducive to timely acquisition and adjustment of the robot's position and posture information in each control stage, thereby gradually improving the positioning accuracy; this scheme realizes precise positioning control of the wheel-foot robot through the cyclic acquisition and correction of multi-dimensional position and posture information, thereby improving the accuracy of wheel-foot robot control.

[0096] In an exemplary embodiment, referring to Figure 6 , according to the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, specifically including the following contents: according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane is determined; according to the angle information and the lateral distance information between the preset position and the center point of the positioning mark, the preset distance information is determined; according to the first distance information, the wheeled robot is controlled to move until the first distance information is within the preset distance range corresponding to the preset distance information.

[0097] The center point of the positioning mark may be a point marked by ArUco in the image, for example, the center point of the positioning mark may be the mark center point K.

[0098] The angle information may be the angle formed by the center point of the camera and the center point of the positioning mark on a preset reference plane. For example, the angle information may be the angle of angle B in triangle ABE.

[0099] The lateral distance information may be the horizontal distance from the preset position to the center point of the positioning mark, for example, the lateral distance information may be a fixed length AB.

[0100] The preset distance range may be an allowable distance error range. For example, when the preset distance information is BE, the preset distance range may be a range from BE-K1 to BE+K1 (where K1 is an error fluctuation range value).

[0101] Optionally, the controller of the wheeled robot calculates the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane YZ plane, that is, angle B, through the Pose array provided by the ArUco positioning mark, according to the image information of the positioning mark in the captured image of the camera of the wheeled robot (for example, the image information of the positioning mark in the captured image of the camera of the wheeled robot can be matched with the image information corresponding to the Pose array), and constructs a right triangle ABE with the angle information and the fixed length AB as known conditions, and calculates the preset distance information BE. Subsequently, the controller of the wheeled robot determines whether the wheeled robot is moving forward or backward according to the difference between the first distance information BC and the preset distance information BE, until the first distance information BC is within the preset distance range corresponding to the preset distance information BE.

[0102] The technical solution provided in this embodiment determines the preset distance information based on the angle information formed by the center point of the camera and the center point of the positioning mark, and the lateral distance information between the preset position and the center point of the positioning mark, which is conducive to establishing an accurate spatial geometric relationship, thereby facilitating improving the positioning accuracy of the wheeled robot; by adjusting the position in a manner that controls the first distance information within the preset distance range, the positioning error can be effectively reduced, thereby ensuring that the wheeled robot can accurately reach the preset position.

[0103] In an exemplary embodiment, the first distance information between the camera and the positioning marker is determined, specifically including the following contents: basic first distance information between the camera and the positioning marker is determined according to the image information of the positioning marker in the captured image of the camera of the wheeled robot; when the basic first distance information is greater than a preset distance threshold, multiple candidate first distance information between the camera and the positioning marker is determined according to the image information of the positioning marker in the captured image of the camera of the wheeled robot; the first distance information is determined according to the average information between the multiple candidate first distance information; according to the image information of the positioning marker in the captured image of the camera of the wheeled robot, the angle information formed by the center point of the camera and the center point of the positioning marker on the preset reference plane is determined, specifically including the following contents: according to the image information of the positioning marker in the captured image of the camera of the wheeled robot, multiple candidate angle information formed by the center point of the camera and the center point of the positioning marker on the preset reference plane is determined; the angle information is determined according to the average information between the multiple candidate angle information.

[0104] Among them, reference Figure 2 and Figure 3 The basic first distance information may be the initial distance data between the camera and the positioning identifier. For example, the basic first distance information may be the BC length first estimated based on the Pose array provided by the ArUco positioning identifier.

[0105] The preset distance threshold may be a distance reference value used to determine whether multiple measurements are required. For example, the preset distance threshold may be a distance limit at which errors may occur in the position and posture data provided by the ArUco positioning identifier at long distances.

[0106] Among them, reference Figure 6 The candidate first distance information may be a single distance data in a distance data set obtained by multiple measurements. For example, the candidate first distance information may be 30 sets of BC side length data collected when the robot is stationary.

[0107] The average information may be an average value obtained after statistical processing of multiple data. For example, the average information may be a result obtained by averaging the remaining data after removing the maximum value and the minimum value.

[0108] Among them, reference Figure 6 The candidate angle information may be a single angle data in a set of angle data obtained by multiple measurements. For example, the candidate angle information may be 30 sets of angle B data collected when the robot is stationary.

[0109] Optionally, refer to Figure 6 , the controller of the wheeled robot estimates the position and posture according to the Pose array provided by the ArUco positioning mark, and calculates the basic first distance information BC between the camera and the positioning mark. When the basic first distance information BC is greater than the preset distance threshold, the controller of the wheeled robot controls the wheeled robot to remain stationary and continuously collects 30 sets of candidate first distance information BC between the camera and the positioning mark. At the same time, the controller of the wheeled robot collects the candidate angle information angle B formed by the center point of the 30 sets of cameras and the center point of the positioning mark on the preset reference plane YZ plane. After obtaining the 30 sets of candidate first distance information BC, the controller of the wheeled robot removes the maximum and minimum values ​​therein, and performs arithmetic averaging on the remaining 28 sets of candidate first distance information BC to obtain the final first distance information BC. Similarly, the controller of the wheeled robot also removes the maximum and minimum values ​​of the 30 sets of candidate angle information angle B, and performs arithmetic averaging on the remaining 28 sets of candidate angle information angle B to obtain the final angle information angle B. This method can effectively reduce the error of the position and attitude data provided by the ArUco positioning marker in long-distance situations, as well as the error caused by the direction slippage that may occur in the wheeled robot during the calibration process.

[0110] The technical solution provided in this embodiment, by collecting multiple candidate first distance information and multiple candidate angle information when the basic first distance information is greater than the preset distance threshold, and averaging these information, is helpful to eliminate the single measurement error caused by the reduction of the ArUco code projection size under long-distance conditions, thereby helping to improve the accuracy of the wheeled robot in long-distance positioning.

[0111] In an exemplary embodiment, after controlling the wheeled robot to move to a preset position according to the second distance information, it also includes the following contents: obtaining the current horizontal coordinate of the center point of the positioning mark in the captured image of the camera of the wheeled robot; determining whether the wheeled robot has successfully moved to the preset position according to the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning mark.

[0112] The current horizontal coordinate may be a real-time horizontal position parameter of the center point of the positioning mark in the pixel coordinate system. For example, the current horizontal coordinate may be the horizontal coordinate Xp of the center point P of the tag.

[0113] Among them, the preset horizontal coordinate can be the standard horizontal position parameter of the center point of the positioning mark obtained during the data calibration stage. For example, the preset horizontal coordinate can be the horizontal coordinate CalibrationValue2 of the center point K of the label obtained when the robot is in the rectangular frame and the midpoint of the robot's face is vertically above point D.

[0114] Optionally, the controller of the wheeled robot establishes a pixel coordinate system in the image, obtains the current horizontal coordinate Xp of the center point P of the positioning marker through the Pose array provided by the ArUco positioning marker, and compares the current horizontal coordinate Xp with the preset horizontal coordinate CalibrationValue2 to determine whether the wheeled robot has successfully moved to the preset position.

[0115] The technical solution provided in this embodiment, by acquiring the current horizontal coordinate of the center point of the positioning mark and comparing it with the preset horizontal coordinate, is conducive to accurately verifying the actual position of the wheeled robot after completing the movement, thereby helping to solve the positioning deviation problem caused by slipping that may occur during the movement of the wheeled robot and ensure that the wheeled robot can accurately reach the preset position.

[0116] In an exemplary embodiment, the wheeled robot is controlled to rotate according to the image information of the positioning marker in the image captured by the camera of the wheeled robot, which specifically includes the following contents: performing image recognition processing on the image captured by the camera of the wheeled robot to obtain the image information of the positioning marker in the image captured by the camera of the wheeled robot; determining the lateral deviation information between the center point of the positioning marker and the center point of the image captured by the camera of the wheeled robot according to the image information of the positioning marker in the image captured by the camera of the wheeled robot; and controlling the wheeled robot to rotate according to the lateral deviation information until the lateral deviation information is within a preset lateral deviation range.

[0117] The image recognition processing may be a process of performing distortion correction and feature recognition on the image captured by the camera, and identifying the ArUco positioning mark.

[0118] The lateral deviation information may be the distance difference between the center point of the positioning mark and the center point of the image captured by the camera in the horizontal direction.

[0119] The preset lateral deviation range may be a maximum value range of the allowable lateral deviation.

[0120] The center point of the captured image may be the center position of the image captured by the camera in the pixel coordinate system. For example, the center point of the captured image may be the pixel position corresponding to the total pixel width x0 / 2 of the image.

[0121] Optionally, the controller of the wheel-foot robot performs recognition processing of the ArUco positioning mark on the corrected image, establishes a pixel coordinate system in the image, calculates half of the total pixel width x0 of the image as the horizontal coordinate of the center point of the image captured by the camera, and simultaneously obtains the horizontal coordinate of the center point of the positioning mark, and obtains the lateral deviation information by calculating the difference between the two horizontal coordinates. After obtaining the lateral deviation information, the controller of the wheel-foot robot determines whether the deviation is within a preset lateral deviation range. If the lateral deviation is positive and exceeds the preset range, the controller of the wheel-foot robot controls the wheel-foot robot to rotate to the right; if the lateral deviation is negative and exceeds the preset range, the controller of the wheel-foot robot controls the wheel-foot robot to rotate to the left; when the lateral deviation enters the preset range, the controller of the wheel-foot robot stops the rotation of the wheel-foot robot. This rotation control method based on lateral deviation can ensure that the positioning mark remains in the central area of ​​the camera's field of view, thereby obtaining the best recognition effect.

[0122] The technical solution provided in this embodiment calculates the lateral deviation information between the center point of the positioning mark and the center point of the image collected by the camera, and controls the rotation movement of the wheeled robot based on the deviation information, which is beneficial to keeping the positioning mark always in the central area of ​​the camera's field of view, thereby facilitating obtaining the best image recognition effect; dynamic alignment between the camera center and the positioning mark center is achieved through lateral deviation control, ensuring the accuracy of the subsequent positioning process.

[0123] In an exemplary embodiment, before controlling the wheeled robot to rotate according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, it also includes the following contents: acquiring the captured image of the camera of the wheeled robot; controlling the wheeled robot to rotate until the captured image of the camera of the wheeled robot contains the positioning mark.

[0124] Optionally, the controller of the wheel-foot robot obtains the captured image and camera parameter information of the camera of the wheel-foot robot. The controller of the wheel-foot robot uses the camera parameters to perform distortion correction processing on the captured image to eliminate barrel distortion or pincushion distortion caused by the camera lens. The controller of the wheel-foot robot controls the wheel-foot robot to rotate at a fixed angular velocity. During the rotation process, the controller of the wheel-foot robot continues to perform recognition processing of the ArUco positioning mark on the captured image of the camera of the wheel-foot robot. When the controller of the wheel-foot robot successfully recognizes the complete ArUco positioning mark in the captured image of the camera of the wheel-foot robot, the controller of the wheel-foot robot stops the rotation action of the wheel-foot robot.

[0125] The technical solution provided in this embodiment, by controlling the wheeled robot to rotate until the captured image of its camera contains the positioning mark, is beneficial to ensuring that the positioning mark has entered the field of view of the camera before performing accurate positioning control, thereby helping to avoid invalid image recognition and position calculation when the positioning mark is not in the field of view, thereby improving the efficiency of the entire positioning process.

[0126] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0127] Based on the same inventive concept, the embodiment of the present application also provides a wheeled robot control device for implementing the wheeled robot control method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more wheeled robot control device embodiments provided below can refer to the limitations of the wheeled robot control method above, and will not be repeated here.

[0128] In an exemplary embodiment, Figure 7 As shown, a wheel-legged robot control device is provided, and the wheel-legged robot control device 700 may include:

[0129] The first control module 701 is used to control the wheeled robot to rotate according to the image information of the positioning mark in the image collected by the camera of the wheeled robot, and determine the first distance information between the camera and the positioning mark;

[0130] The second control module 702 is used to control the movement of the wheeled robot according to the first distance information and the preset distance information between the camera and the positioning mark, and determine the angle information between the wheeled robot and the preset reference plane according to the image information of the positioning mark in the image collected by the camera of the wheeled robot;

[0131] The third control module 703 is used to control the wheeled robot to rotate according to the angle information, and determine the second distance information between the camera and the positioning mark according to the image information of the positioning mark in the image collected by the camera of the wheeled robot;

[0132] The fourth control module 704 is used to control the wheeled robot to move to a preset position according to the second distance information.

[0133] In an exemplary embodiment, the second control module 702 is also used to determine the angle information formed by the center point of the camera and the center point of the positioning marker on a preset reference plane based on the image information of the positioning marker in the captured image of the camera of the wheeled robot; determine the preset distance information based on the angle information and the lateral distance information between the preset position and the center point of the positioning marker; and control the movement of the wheeled robot based on the first distance information until the first distance information is within the preset distance range corresponding to the preset distance information.

[0134] In an exemplary embodiment, the first control module 701 is also used to determine the basic first distance information between the camera and the positioning marker based on the image information of the positioning marker in the captured image of the camera of the wheeled robot; when the basic first distance information is greater than a preset distance threshold, multiple candidate first distance information between the camera and the positioning marker is determined based on the image information of the positioning marker in the captured image of the camera of the wheeled robot; the first distance information is determined based on the average information between the multiple candidate first distance information; the second control module 702 is also used to determine multiple candidate angle information formed by the center point of the camera and the center point of the positioning marker on a preset reference plane based on the image information of the positioning marker in the captured image of the camera of the wheeled robot; the angle information is determined based on the average information between the multiple candidate angle information.

[0135] In an exemplary embodiment, the device 700 also includes: a coordinate acquisition module for acquiring the current horizontal coordinate of the center point of the positioning mark in the captured image of the camera of the wheeled robot; and determining whether the wheeled robot has successfully moved to the preset position based on the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning mark.

[0136] In an exemplary embodiment, the first control module 701 is also used to perform image recognition processing on the image captured by the camera of the wheel-foot robot to obtain image information of the positioning mark in the image captured by the camera of the wheel-foot robot; determine the lateral deviation information between the center point of the positioning mark and the center point of the image captured by the camera of the wheel-foot robot based on the image information of the positioning mark in the image captured by the camera of the wheel-foot robot; and control the wheel-foot robot to rotate based on the lateral deviation information until the lateral deviation information is within a preset lateral deviation range.

[0137] In an exemplary embodiment, the device 700 further includes: an image acquisition module, configured to acquire an image captured by a camera of the wheeled robot; and control the wheeled robot to rotate until the image captured by the camera of the wheeled robot contains a positioning mark.

[0138] Each module in the above wheeled robot control device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0139] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a wheeled robot control method is realized. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0140] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0142] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0143] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A wheeled robot control method, characterized in that: The method comprises: According to the image information of the positioning mark in the image collected by the camera of the wheeled robot, the wheeled robot is controlled to rotate, and the first distance information between the camera and the positioning mark is determined; According to the first distance information and the preset distance information between the camera and the positioning mark, the wheeled robot is controlled to move, and according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the angle information between the wheeled robot and the preset reference plane is determined; According to the angle information, the wheeled robot is controlled to rotate, and according to the image information of the positioning mark in the image captured by the camera of the wheeled robot, the second distance information between the camera and the positioning mark is determined; According to the second distance information, the wheeled robot is controlled to move to a preset position.

2. The method according to claim 1, characterized in that The controlling the movement of the wheeled robot according to the first distance information and the preset distance information between the camera and the positioning mark includes: According to the image information of the positioning mark in the captured image of the camera of the wheeled robot, determine the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane; Determining the preset distance information according to the angle information and the lateral distance information between the preset position and the center point of the positioning mark; According to the first distance information, the wheeled robot is controlled to move until the first distance information is within a preset distance range corresponding to the preset distance information.

3. The method according to claim 2, characterized in that The determining of the first distance information between the camera and the positioning mark includes: Determine basic first distance information between the camera and the positioning mark according to image information of the positioning mark in the image collected by the camera of the wheeled robot; When the basic first distance information is greater than a preset distance threshold, a plurality of candidate first distance information between the camera and the positioning mark is determined according to the image information of the positioning mark in the captured image of the camera of the wheeled robot; Determining the first distance information according to average information among the plurality of candidate first distance information; The method of determining the angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane based on the image information of the positioning mark in the captured image of the camera of the wheeled robot includes: According to the image information of the positioning mark in the captured image of the camera of the wheeled robot, determine a plurality of candidate angle information formed by the center point of the camera and the center point of the positioning mark on the preset reference plane; The angle information is determined according to average information among the plurality of candidate angle information.

4. The method according to claim 1, characterized in that: After controlling the wheeled robot to move to a preset position according to the second distance information, the method further includes: Obtaining the current horizontal coordinate of the center point of the positioning mark in the captured image of the camera of the wheeled robot; According to the current horizontal coordinate and the preset horizontal coordinate of the center point of the positioning mark, it is determined whether the wheeled robot has successfully moved to the preset position.

5. The method according to claim 1, characterized in that The step of controlling the wheeled robot to rotate according to the image information of the positioning mark in the image collected by the camera of the wheeled robot comprises: Performing image recognition processing on the image collected by the camera of the wheeled robot to obtain image information of the positioning mark in the image collected by the camera of the wheeled robot; Determine, according to image information of the positioning mark in the image captured by the camera of the wheeled robot, lateral deviation information between the center point of the positioning mark and the center point of the image captured by the camera of the wheeled robot; According to the lateral deviation information, the wheeled robot is controlled to rotate until the lateral deviation information is within a preset lateral deviation range.

6. The method according to any one of claims 1 to 5, characterized in that Before controlling the wheeled robot to rotate according to the image information of the positioning mark in the captured image of the camera of the wheeled robot, the method further includes: Acquire an image captured by a camera of the wheeled robot; The wheeled robot is controlled to rotate until the image captured by the camera of the wheeled robot contains the positioning mark.

7. A wheel-foot robot control device, characterized in that: The device comprises: A first control module is used to control the wheeled robot to rotate according to image information of the positioning mark in the image collected by the camera of the wheeled robot, and determine first distance information between the camera and the positioning mark; a second control module, configured to control the movement of the wheeled robot according to the first distance information and the preset distance information between the camera and the positioning mark, and determine the angle information between the wheeled robot and a preset reference plane according to the image information of the positioning mark in the image captured by the camera of the wheeled robot; a third control module, configured to control the wheeled robot to rotate according to the angle information, and determine second distance information between the camera and the positioning mark according to image information of the positioning mark in an image captured by a camera of the wheeled robot; The fourth control module is used to control the wheeled robot to move to a preset position according to the second distance information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images