A virtual grid-based ranging method and cleaning robot

Abstract

The application discloses a virtual grid-based distance measuring method and a cleaning robot, and relates to the technical field of intelligent household appliances. The virtual grid-based distance measuring method comprises the following steps: acquiring a pose inclination angle of the cleaning robot in a current pose state and target image data collected for a target object; calculating a current blind area length based on a preset blind area length calculation model and the pose inclination angle, and obtaining a distance difference between the current blind area length and a known initial blind area length; determining target pixel coordinates of the target object in a virtual grid based on the target image data and preset virtual grid data; and determining an actual detection distance between the target object and the cleaning robot based on the distance difference, the target pixel coordinates and a preset distance matrix table. Therefore, the application has the advantages of accurate detection, cost saving and strong adaptability.

Description

Technical Field

[0001] This application relates to the technical field of smart home appliances, and more specifically, to a ranging method based on a virtual grid and a cleaning robot. Background Technology

[0002] In existing technologies, cleaning robots typically use laser ranging components to detect the actual distance between themselves and surrounding obstacles. However, this ranging method is only suitable for determining whether avoidance is necessary based on actual distance and cannot identify the type of obstacle or the actual position of its bottom. Therefore, cleaning robots based on visual recognition technology have emerged. These robots usually use binocular cameras or 3D ToF technology to complete the ranging task. However, this method is costly, has a small field of view, and suffers from blind spots, which can easily lead to significant errors in actual ranging. Summary of the Invention

[0003] The purpose of this application is to provide a distance measurement method and a cleaning robot based on a virtual grid. By using a gyroscope and a camera to detect real-time data and in conjunction with a distance matrix table, the accuracy of distance measurement and obstacle avoidance performance of the cleaning robot during operation are improved.

[0004] The embodiments of this application are implemented as follows:

[0005] The first aspect of this application provides a ranging method based on a virtual mesh. The method includes: acquiring the pose tilt angle of a cleaning robot in its current pose state and target image data collected for a target object; calculating the current blind zone length based on a preset blind zone length calculation model and the pose tilt angle, and obtaining the distance difference between the current blind zone length and the known initial blind zone length; determining the target pixel coordinates of the target object in the virtual mesh based on the target image data and preset virtual mesh data; and determining the actual detection distance between the target object and the cleaning robot based on the distance difference, the target pixel coordinates, and a preset distance matrix table.

[0006] In one embodiment, determining the actual detection distance between the target object and the cleaning robot based on the distance difference, the target pixel coordinates, and a preset distance matrix table includes: calibrating the detection distance corresponding to each pixel coordinate in the preset distance matrix table based on the distance difference; and determining the actual detection distance between the target object and the cleaning robot based on the target pixel coordinates and the calibrated distance matrix table.

[0007] In one embodiment, the actual detection distance between the target object and the cleaning robot is determined based on the distance difference, the target pixel coordinates, and a preset distance matrix table. This includes: determining the initial detection distance between the target object and the cleaning robot based on the target pixel coordinates and the detection distances corresponding to each pixel coordinate in the preset distance matrix table; and calculating the actual detection distance between the target object and the cleaning robot based on the distance difference and the initial detection distance.

[0008] In one embodiment, the current blind zone length is calculated based on a preset blind zone length calculation model and a pose tilt angle, and the distance difference between the current blind zone length and the known initial blind zone length is obtained. This includes: calculating the total tilt angle of the camera based on the pose tilt angle and the camera's mounting tilt angle; inputting the total tilt angle into the preset blind zone length calculation model to obtain the current blind zone length; and calculating the distance difference between the current blind zone length and the known initial blind zone length.

[0009] In one embodiment, before calculating the distance difference between the current blind zone length and the known initial blind zone length, the method further includes: inputting the installation tilt angle into a preset blind zone length calculation model to obtain the known initial blind zone length.

[0010] In one embodiment, before calculating the current blind zone length based on a preset blind zone length calculation model and pose tilt angle, the blind zone length calculation model is obtained by the following method: acquiring multiple sets of test data, including the camera installation height, the cleaning robot pose tilt angle and the corresponding blind zone length; fitting the numerical relationship curve between the installation height, pose tilt angle and the corresponding blind zone length based on the multiple sets of test data, and establishing the blind zone length calculation model.

[0011] In one embodiment, before determining the target pixel coordinates of the target object in the virtual grid based on the target image data and the preset virtual grid data, the method further includes: calculating the installation tilt angle of the camera based on the installation height of the camera, the preset vertical field of view and the length of the detection blind zone; determining the detection distance corresponding to all pixels in the virtual grid at the installation tilt angle based on the pixel coordinates of each intersection point in the virtual grid and the detection distance corresponding to each intersection point, and storing it as a distance matrix table.

[0012] In one embodiment, the detection distances corresponding to all pixels in the virtual grid at the installation tilt angle are determined based on the pixel coordinates of each intersection point in the virtual grid and the detection distances corresponding to each intersection point. This includes: calculating the actual relative distance between a pixel and its adjacent intersection points based on the pixel coordinates of each pixel in the virtual grid and its adjacent intersection points; and calculating the detection distances corresponding to all pixels in the virtual grid based on the detection distances corresponding to the adjacent intersection points and the actual relative distances.

[0013] In one embodiment, before determining the target pixel coordinates of the target object in the virtual grid based on the target image data and the preset virtual grid data, the method further includes: adjusting the installation tilt angle of the camera based on a preset unit angle value; and when the pose tilt angle of the cleaning robot is 0, determining the detection distances corresponding to all pixels in the virtual grid under different installation tilt angles.

[0014] A second aspect of this application provides a cleaning robot, which includes a gyroscope, at least one camera, and a processor. The gyroscope is used to detect the tilt angle of the cleaning robot; the camera is used to acquire target images; the processor is connected to the gyroscope and the camera, and is configured to execute the virtual mesh-based ranging method provided in any embodiment of the first aspect of this application.

[0015] The advantages of this application compared to the prior art are:

[0016] This application addresses the problem of poor obstacle avoidance performance in existing cleaning robots due to inaccurate distance measurement. It acquires the current pose of the cleaning robot and relevant data from the image in front of it using a gyroscope and camera. This data, combined with a pre-set distance matrix table and a blind zone length calculation model, outputs a calibration deviation value, thereby calibrating the robot's detection distance. This application effectively improves the distance detection accuracy of cleaning robots while significantly reducing manufacturing costs. Furthermore, the improved obstacle avoidance performance makes the robot safer and more efficient, ultimately extending its lifespan and enhancing the user experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of a cleaning robot provided in one embodiment of this application;

[0019] Figure 2 A schematic flowchart of a virtual grid-based ranging method provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a calibration plate with fixed dimensional accuracy provided in an embodiment of this application; wherein, Figure 3 (a) is a schematic diagram of a chessboard calibration plate provided in an embodiment of this application; Figure 3(b) is a schematic diagram of a dot calibration plate provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram illustrating an application scenario for determining the camera installation tilt angle according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram illustrating the extraction of lines and corner points from a chessboard grid according to an embodiment of this application; wherein, Figure 5 (a) is a schematic diagram before the lines and corners of the chessboard are extracted; Figure 5 (b) is a schematic diagram after extracting the lines and corner points of the chessboard grid;

[0023] Figure 6 This is a schematic diagram of a virtual mesh provided in an embodiment of this application; wherein, Figure 6 (a) A schematic diagram illustrating the determination of pixel coordinates for any point P within a virtual mesh, according to an embodiment of this application; Figure 6 (b) is a schematic diagram of a circular virtual mesh provided in an embodiment of this application; Figure 6 (c) A schematic diagram of a virtual mesh composed of triangles and quadrilaterals provided in an embodiment of this application;

[0024] Figure 7 This is a schematic diagram illustrating the relationship between the blind zone length and the pose tilt angle of a cleaning robot according to an embodiment of this application. Figure 7 (a) is a schematic diagram of the blind zone length of the cleaning robot when it is tilted at no position angle; Figure 7 (b) is a schematic diagram of the blind zone length of the cleaning robot when it has a tilt angle.

[0025] Figure 8 This is a schematic diagram of a blind zone length fitting curve provided in an embodiment of this application;

[0026] Figure 9 A schematic diagram illustrating an application scenario of the virtual grid-based ranging method provided in an embodiment of this application;

[0027] Figure 10 This is a schematic diagram illustrating distance measurement based on the coordinates of the lower edge of a target object in front of an image, as provided in an embodiment of this application.

[0028] Reference numerals: 1-Cleaning robot; 10-Bus; 11-Processor; 12-Memory; 13-Gyroscope; 14-Camera; 2-Walking plane; 20-Horizontal baseline; 21-Upper boundary; 22-Lower boundary; 23-Optical axis; 3-Target object. Detailed Implementation

[0029] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0030] Similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] The technical solution of this application will now be clearly and completely described with reference to the accompanying drawings.

[0032] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a cleaning robot 1 provided in an embodiment of this application, as shown below. Figure 1 As shown, the cleaning robot 1 includes a gyroscope 13, at least one camera 14, at least one processor 11, and a memory 12. Figure 1 Taking a processor 11 as an example, the processor 11 and the memory 12 are connected via a bus 10. The processor 11 is also connected to a gyroscope 13 and a camera 14. The memory 12 stores instructions that can be executed by at least one processor 11. The instructions are executed by at least one processor 11 to cause at least one processor 11 to perform a virtual mesh-based ranging method as described in the following embodiments. The gyroscope 13 is used to detect the tilt angle of the cleaning robot 1; the camera 14 is used to acquire target images.

[0033] In one embodiment, the cleaning robot 1 is also equipped with a laser rangefinder, which is installed on the body next to the camera 14 or on the camera 14, and is used to detect the height distance between the center of the camera and the walking plane 2 on which the cleaning robot 1 is located.

[0034] The cleaning robot provided in this application can acquire images of a target object in front of it using a camera. Based on the position of the target object in the image and the correlation between the actual position of each pixel in the pre-stored image and the target object's position, it determines the actual relative distance between the target object and the cleaning robot on a horizontal plane in two mutually perpendicular directions (denoted as the horizontal and vertical directions). Furthermore, the images acquired by the cleaning robot's camera have blind spots, and due to the camera's installation tilt and the tilt caused by varying road conditions during operation, the total tilt angle corresponding to the camera's optical axis differs. Consequently, the length of the blind spot in the image acquired by the camera varies depending on the robot's movement. Therefore, the cleaning robot of this application can detect and calculate a more accurate actual relative distance between the cleaning robot and the target object in front of it under different motion environments, taking into account factors such as the position of the target object in the acquired image and the total tilt angle of the camera's optical axis.

[0035] In one embodiment of this application, since a monocular camera is more effective for visual monitoring of AI recognition or cleaning robots, the cleaning robot has this camera installed on its body before leaving the factory. The monocular camera, combined with a gyroscope and laser rangefinder, enables the calculation of the height between the cleaning robot and the walking plane, the calculation of the robot's pitch angle, and the acquisition of images of objects in front, thereby further improving the accuracy of detecting the actual relative distance between the target object and the cleaning robot. This application, while saving costs, can simultaneously achieve object detection, recognition, scene monitoring, and obstacle avoidance distance measurement using a monocular camera.

[0036] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating a virtual grid-based ranging method provided in an embodiment of this application. Figure 2 As shown, the ranging method based on a virtual grid includes the following steps:

[0037] S110: Acquire the tilt angle of the cleaning robot in its current pose state and the target image data collected for the target object.

[0038] The current pose state refers to the different walking postures of a cleaning robot during its movement, due to variations in the road conditions (e.g., a smooth, flat surface or surfaces with potholes, bumps, etc.). In this application, to achieve distance measurement of objects ahead, the cleaning robot typically uses a gyroscope to detect the tilt angle to characterize its current pose state. The walking surface is a horizontal surface.

[0039] The cleaning robot acquires attitude data from a gyroscope during real-time operation, including yaw angle, pitch angle, and roll angle. Since the camera in this application is located directly in front of the cleaning robot's body, the pitch angle is used as the pose tilt angle to evaluate the camera's attitude.

[0040] In this step, the cleaning robot collects images of the target object in front of it and its own posture data in real time during operation. It also obtains target image data and posture tilt angle from the camera and gyroscope. In subsequent steps, it queries and calibrates the distance between the target object in front of it and the cleaning robot, and obtains the actual detection distance.

[0041] S120: Based on the preset blind zone length calculation model and pose tilt angle, calculate the current blind zone length and obtain the distance difference between the current blind zone length and the known initial blind zone length.

[0042] In this step, the cleaning robot can determine the actual tilt angle of the camera by the pose tilt angle. After inputting the actual tilt angle of the camera into the blind zone length calculation model, the robot calculates the distance difference based on the current blind zone length output by the blind zone length calculation model and the initial blind zone length of the camera when the pose tilt angle is 0. This distance difference is then used to calibrate the queried detection distance in subsequent steps.

[0043] S130: Based on the target image data and the preset virtual grid data, determine the target pixel coordinates of the target object in the virtual grid.

[0044] In this step, the cleaning robot confirms the pixel position (pixel coordinates) of the target object in the image based on the target image data, and determines the position of the target object in the virtual grid based on the pre-set virtual grid, and represents it by the target pixel coordinates.

[0045] S140: Based on the distance difference, target pixel coordinates, and a preset distance matrix table, determine the actual detection distance between the target object and the cleaning robot.

[0046] In this step, the cleaning robot can initially determine the distance between the target object and the cleaning robot based on a preset distance matrix table and the determined target pixel coordinates. This initially determined distance corresponds to the distance when the camera's tilt angle is 0 degrees, and deviates somewhat from the actual distance. This deviation is mainly caused by the camera's tilt angle, which alters the length of the corresponding blind zone in the captured image. Therefore, based on the distance difference corresponding to the tilt angle, the cleaning robot can effectively calibrate the initially determined distance or the preset distance matrix table, and then determine the calibrated actual detection distance to improve the accuracy of distance detection.

[0047] Figure 3 This is a schematic diagram of a calibration plate with fixed dimensional accuracy provided in an embodiment of this application; wherein, Figure 3 (a) is a schematic diagram of a chessboard calibration plate provided in an embodiment of this application; Figure 3 (b) is a schematic diagram of a dot calibration plate provided in an embodiment of this application. Before the cleaning robot leaves the factory, it must first be identified and processed based on a preset physical calibration plate, so as to generate and store a virtual grid and related data according to the relevant image data, and apply the virtual grid to actual distance measurement after leaving the factory.

[0048] Therefore, operators need to prepare in advance, such as Figure 3 (a) Figure 3(b) shows a calibration plate with fixed dimensional accuracy. The dimensional accuracy Rt of the calibration plate can be 0.1-1cm. The calibration plate uses a relatively regular outer contour shape and marking points, such as a dot target, a dot square calibration plate, a checkerboard, etc. Considering the special nature of the scene, the dots can be replaced with cross-shaped marks. The outer contour shape of the calibration plate can also be a shape other than square or circle.

[0049] Before leaving the factory, it is also necessary to determine the installation tilt angle of the camera mounted on the device due to installation deviation, generate and store the distance matrix table corresponding to the installation tilt angle based on the virtual grid, and establish a blind zone length calculation model based on multiple sets of measured data. Therefore, the virtual grid-based ranging method includes the following steps S210 to S270.

[0050] S210: Calculate the camera's installation tilt angle based on the camera's installation height, preset vertical field of view, and detection blind zone length.

[0051] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating an application scenario for determining the installation tilt angle of the camera 14 according to an embodiment of this application. Figure 4 As shown, the monocular camera 14 installed on the cleaning robot 1 has a resolution greater than 200W pixels and a vertical field of view (Fov) greater than 58°, where the vertical field of view includes the upper boundary 21 and the lower boundary 22. After the camera 14 is installed, the height of the camera 14 above the ground is expressed as the installation height CH (here, the vertical distance between the center of the camera 14 and the walking plane 2 (horizontal ground) is used as the standard; the installation height CH can be measured manually or detected by a laser rangefinder). The vertical distance between the lower boundary 22 of the imaging boundary of the camera 14 and the camera 14 is the blind zone length CDis.

[0052] In this step, the cleaning robot 1 determines the installation tilt angle of the optical axis 23 of the camera 14 relative to the horizontal reference line 20 based on the installation height CH, the blind zone length CDis, and the vertical field of view Fov. The calculation formula is as follows:

[0053]

[0054] Among them, the installation tilt angle The value of needs to be noted:

[0055] when At that time, the installation tilt angle was downward.

[0056] when At that time, the installation tilt angle was upward.

[0057] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the extraction of lines and corner points from a chessboard grid according to an embodiment of this application; wherein, Figure 5 (a) is a schematic diagram before the lines and corners of the chessboard are extracted; Figure 5 (b) is a schematic diagram after extracting the lines and corners of the chessboard.

[0058] Before generating the distance matrix table based on the virtual mesh data, the cleaning robot needs to collect images of the actual calibration board and extract lines, vertices, or corners based on the collected image data to further generate virtual mesh data.

[0059] like Figure 5 (a) Figure 5 As shown in (b), the cleaning robot first performs corner detection on the checkerboard calibration board, extracting all visible corners within its field of view. Considering the actual distance measurement requirements, the cleaning robot selects nH horizontal corner sequence numbers and fits them into nH straight lines; it also selects nV vertical angle sequence numbers and fits them into nV oblique straight lines that actually represent the vertical direction.

[0060] The horizontal field of view (Hl) and vertical field of view (Vl) of the sweeper are proportional to the width of the machine body, and are denoted as follows:

[0061] Hl = 2R + α;

[0062] Vl=2R+β;

[0063] Where 2R is the fuselage diameter or fuselage width, α and β are adjusted according to the actual scene, and a value of 0.5R is recommended. After determining the horizontal and vertical field of view lengths, the number of corner points nH and nV in the horizontal and vertical directions are calculated according to the following formula:

[0064]

[0065]

[0066] Based on the nH×nV corner points obtained in the above steps, the cleaning robot records the pixel coordinates of each corner point as (i, j), and takes the i-th... The vertical oblique line (i.e. the line where the camera is located) is the Y-axis line where the origin of the coordinate system is located. Therefore, the horizontal coordinate to the left of the Y-axis line is negative, and the horizontal coordinate to the right of the line is positive.

[0067] The formula for calculating the actual coordinates (Rx, Ry) of each corner point, corresponding to the pixel coordinate index (i, j), is as follows:

[0068] Rx = i * Rt;

[0069] Ry = j * Rt;

[0070] Where i and j correspond to the i-th horizontal line and the j-th vertical line, respectively. The cleaning robot starts counting from the origin coordinates and calculates the actual coordinate values ​​of all corner points.

[0071] S220: Based on the pixel coordinates of each intersection point in the virtual mesh and the corresponding detection distance, determine the detection distance corresponding to all pixels in the virtual mesh at the installation tilt angle, and store it as a distance matrix table. This step includes the following steps S221 to S223.

[0072] S221: Construct a virtual mesh;

[0073] Please see Figure 6 (a) to Figure 6 (c), Figure 6 This is a schematic diagram of a virtual mesh provided in an embodiment of this application; wherein, Figure 6 (a) A schematic diagram illustrating the determination of pixel coordinates for any point P within a virtual mesh, according to an embodiment of this application; Figure 6 (b) is a schematic diagram of a circular virtual mesh provided in an embodiment of this application; Figure 6 (c) is a schematic diagram of a virtual mesh composed of triangles and quadrilaterals provided in an embodiment of this application.

[0074] Based on the extraction of diagonal points and the calculation of their corresponding actual coordinates in the above steps, the cleaning robot further extends a virtual mesh. First, a virtual dot matrix is ​​constructed based on the extracted intersection points, with nH×nV intersection points. Then, a virtual mesh with (nH-1)×(nV-1) unit grids is formed based on the virtual dot matrix, with each unit grid having four intersection points, denoted as A, B, C, and D.

[0075] S222: Calculate the actual relative distance between a pixel and its adjacent intersection points based on the pixel coordinates of each pixel in the virtual grid and its adjacent intersection points;

[0076] S223: Based on the detection distance and actual relative distance of adjacent intersection points, calculate the detection distance of each pixel in the virtual grid.

[0077] In the above steps, the cleaning robot calculates the actual coordinates of any point within each grid cell, generating the corresponding actual distance matrix. Given that in practical applications of virtual grids, the target object will not necessarily fall exactly on the four vertices of each grid cell; most will fall inside the grid or on its edges. Therefore, this application pre-calculates and stores the actual coordinates and detection distance of any point within each grid cell as a corresponding distance matrix table, saving computation time for distance measurement after delivery. Subsequently, the cleaning robot only needs to obtain the pixel coordinates of the lower edge of the target object based on the target image and query the corresponding distance matrix.

[0078] The method for calculating the actual relative distance between any pixel within a cell grid and its adjacent intersection points is as follows: Point P is any point within the cell grid, and the coordinates of the four intersection points of the cell grid are A, B, C, and D. Straight lines are drawn through point P in both the horizontal and vertical directions, intersecting the edges of the cell grid at points P1, P2, P3, and P4, respectively. This application determines the actual relative distance between point P and its adjacent intersection points by calculating the pixel coordinates of the four intersection points P1, P2, P3, and P4.

[0079] like Figure 6 As shown in (a), based on the preset checkerboard calibration board and the characteristics of the corresponding captured images, each point on the calibration board in the same horizontal direction corresponds to a pixel in the virtual grid, and they are at the same height coordinate value (i.e., the same vertical Y-coordinate value), that is:

[0080] Y P3 =Y P4 =YP;

[0081] Y P1 =Y A =Y B ;

[0082] Y P2 =Y C =Y D .

[0083] Since the lines connecting the corner points on the checkerboard calibration board correspond to the grid lines in the virtual grid as oblique straight lines, it is necessary to further calculate the slope K of the vertical oblique line where point P is located. P The calculation formula is as follows:

[0084] K P =(X P -X A )K AD +(X B -X P )K BC ;

[0085] In special cases, such as when the adjacent lines AD or BC of the cell containing point P are perpendicular to the horizontal direction, the slope of the line containing point P can be directly set to the slope of another adjacent oblique line on the cell that is not perpendicular to the horizontal direction.

[0086] The X and Y coordinates of point P and its adjacent intersections can be determined based on pixel data in the target image data and the position of point P relative to its adjacent intersections. For example, a cleaning robot can determine the pixel coordinates of adjacent intersection A through the above steps, and then determine the X and Y pixel coordinates of point P based on the pixel data of point P and point A and the position of point P relative to point A. PY P .

[0087] After determining the slope of the straight line containing point P, the slope B of the straight line containing point P can be calculated using the formula for solving the slope of a straight line: Y = KX + B. P For Y P -K P X P Based on this, the x-coordinates of the intersection points are calculated using the coordinates of point P. P1 X P2 X P3 X P4 .

[0088] After determining the horizontal and vertical coordinates of adjacent intersections, the cleaning robot uses the actual horizontal coordinates of points A and B to determine whether the cell grid where point P is located is to the left or right of the camera. When R Ax <0 or R Bx When <0, the corresponding cell grid is located to the left of the camera, and the lower edge of the corresponding target object is also located to the left of the camera.

[0089] The calculation method for the lower edge of the target object and the actual horizontal coordinates of the cleaning robot differs depending on whether the target object is on the left or right side of the camera.

[0090] Firstly, for the pixels on the left, it is necessary to calculate the actual relative distance between PP4 and BC. The calculation formula is as follows:

[0091]

[0092] Therefore, the actual horizontal coordinates of point P on the left side of the camera are:

[0093] R Px =R Bx -R PP4 .

[0094] Secondly, for the pixels on the right, it is necessary to calculate the actual relative distance between P3P and AD. The calculation formula is as follows:

[0095]

[0096] Therefore, the actual horizontal coordinates of point P on the right side of the camera are:

[0097] R Px =R Ax +R P3P .

[0098] Furthermore, in the vertical direction, the actual relative distance between point P and CD is:

[0099]

[0100] Therefore, the vertical coordinates of point P are:

[0101] R Py =R Cy +R PP2 .

[0102] Among them, PixPP4, PixP3P4, PixPP2, and PixP1P2 are the pixel distances of the corresponding pixels. Based on the actual horizontal coordinate value corresponding to point P, the cleaning robot can determine the horizontal detection distance and actual direction of the pixel (whether the target object corresponding to the pixel is located to the left or right of the camera). Based on the vertical coordinate value corresponding to point P and the blind zone length, the cleaning robot can calculate the actual vertical coordinate value corresponding to point P, and then determine the vertical detection distance of the pixel.

[0103] Using the above method, the cleaning robot can calculate the actual direction and detection distance for each pixel in the virtual grid, and store them as a distance matrix table. The detection distance includes both horizontal and vertical detection distances. The distance matrix table can also be directly represented by the actual horizontal and vertical coordinate values ​​of each pixel.

[0104] Please see Figure 6 (b) Figure 6 (c) In this application, the virtual mesh can also be calibrated using a ring. A virtual mesh is formed by arranging distance points in a ring shape. The calibration principle is the same as the mesh calibration method. The cleaning robot extracts the corner points arranged in an arc shape on the calibration board and forms a point matrix, then generates the virtual mesh and the distance matrix table corresponding to the pixels in the virtual mesh using the same method. Similarly, the virtual mesh can also be calibrated using a calibration board combining triangles and quadrilaterals. Then, the corresponding distance matrix table is generated by calculating the intersection points of the quadrilaterals and triangles, and by filling the internal points of the unit mesh formed by adjacent intersection points.

[0105] In one embodiment, after adjusting the camera's mounting tilt angle based on a preset unit angle value, the cleaning robot can also generate multiple distance matrix tables corresponding to different mounting tilt angles of its own camera. That is, the cleaning robot performs multiple sets of calibrations on the mounting tilt angles during the factory stage, generating a corresponding distance matrix table for each unit angle value. The camera's mounting tilt angle can be manually adjusted by an operator based on the unit angle value, or the cleaning robot can control the camera to automatically tilt based on the unit angle value.

[0106] In the above process, when the pose tilt angle is 0, the cleaning robot should determine the detection distance of all pixels in the virtual grid at different installation tilt angles of the camera, and store them as multiple distance matrix tables corresponding to the installation tilt angles. This will allow for matching the installation tilt angles in subsequent applications, reducing the detection distance error caused by different cleaning robots corresponding to cameras at different installation tilt angles querying the same distance matrix table.

[0107] S230: Establish a blind zone length calculation model; this step includes the following sub-steps S231 to S232.

[0108] S231: Acquire multiple sets of test data, including the camera's installation height, the cleaning robot's tilt angle, and the corresponding blind zone length;

[0109] S232: Based on multiple sets of test data, fit the numerical relationship curves between installation height, tilt angle and corresponding blind zone length, and establish a blind zone length calculation model.

[0110] Please see Figure 7 (a) to Figure 7 (b), Figure 7 This is a schematic diagram illustrating the relationship between the blind zone length and the pose tilt angle of a cleaning robot 1 according to an embodiment of this application. Figure 7 (a) is a schematic diagram of the blind zone length of cleaning robot 1 when it has no tilt angle; Figure 7 (b) is a schematic diagram of the blind zone length of the cleaning robot 1 when it has a tilt angle.

[0111] During real-time operation, cleaning robot 1 acquires attitude data from a gyroscope (not shown in the diagram). Since camera 14 is directly in front of the robot, the pitch angle is used to assess the attitude of camera 14 (which is also the attitude of cleaning robot 1). The pitch angle is denoted as the pose tilt angle. Because camera 14 has an installation tilt angle during installation. Therefore, when the cleaning robot 1 encounters uneven ground or potholes during its movement, the actual total tilt angle of the camera 14 is:

[0112]

[0113] Depend on Figure 7 (a) to Figure 7 (b) It can be seen that the camera installation height CH and the camera blind zone length CDis have the following relationship:

[0114]

[0115] In this case, θ is positive when it is tilted upwards and negative when it is tilted downwards.

[0116] Under ideal conditions, the installation tilt angle of cleaning robot 1 At this moment, the posture of camera 14 is completely aligned with that of gyroscope 13, and the tilt angle is... This is the total tilt angle θ of the camera.

[0117] Please see Figure 8 , Figure 8 This is a schematic diagram of a blind zone length fitting curve provided in an embodiment of this application. Figure 8 As shown, the installation tilt angle is ignored. To assess the impact of the blind spot length, the cleaning robot first acquired multiple sets of test data, including the camera's installation height (CH) and the cleaning robot's tilt angle. The corresponding blind zone length CDis is then used. Multiple sets of test data are then fitted to obtain the corresponding relationship curve and the blind zone length calculation model corresponding to the curve. The algorithm formula for the blind zone length calculation model is as follows:

[0118] CDis = 0.056 * θ 2 +0.968*θ+8.865+α*CH

[0119] Where α is the height relationship coefficient; the installation height CH can be obtained by a laser rangefinder.

[0120] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating an application scenario of the virtual grid-based ranging method provided in an embodiment of this application. For example... Figure 9 As shown, after the cleaning robot has generated and stored at least one distance matrix table and a blind spot length calculation model, it can use the distance matrix table to look up the initial detection distance corresponding to the target object in front during actual application after leaving the factory. Then, it inputs the detected installation height, pose tilt angle, and installation tilt angle into the blind spot length calculation model to calibrate the initial detection distance based on the value output by the blind spot length calculation model, thus obtaining the final actual detection distance. Please refer to the steps below for details.

[0121] S240: Acquire the tilt angle of the cleaning robot in its current pose state and the target image data collected for the target object.

[0122] This step is similar to step S110 above. For details, please refer to step S110, which will not be repeated here.

[0123] S250: Based on the preset blind zone length calculation model and pose tilt angle, calculate the current blind zone length and obtain the distance difference between the current blind zone length and the known initial blind zone length.

[0124] In this step, the cleaning robot 1 first uses the pose tilt angle read by the gyroscope and the pre-stored mounting tilt angle of the camera 14. Calculate the total tilt angle θ of camera 14. Please combine this with... Figure 7 (a) to Figure 7 (b) When the cleaning robot 1 encounters uneven ground or potholes during its movement, the actual total tilt angle of camera 14 is:

[0125]

[0126] in, To determine the installation tilt angle; The tilt angle is the positional tilt angle.

[0127] After calculating the total tilt angle of camera 14, the cleaning robot inputs the tilt angle into a preset blind zone length calculation model to obtain the initial blind zone length CDis0. The cleaning robot also inputs the total tilt angle into the preset blind zone length calculation model to obtain the current blind zone length. After obtaining the current blind zone length CDis' and the initial blind zone length CDis0 output by the length calculation model, the cleaning robot calculates the distance difference between the two. This distance difference is used in subsequent steps to calibrate the distance matrix table or the initial detection distance retrieved from the distance matrix table. The formula for calculating the distance difference is as follows:

[0128] ΔCDis=CDis'-CDis0.

[0129] In another embodiment, the cleaning robot can also obtain the pose tilt angle of the gyroscope in the actual scene, ignore the installation tilt angle of the camera, and directly input the pose tilt angle into the blind zone length calculation model to calculate the current blind zone length of the camera, which is the actual vertical distance between the lower edge of the image and the camera; then define the pose tilt angle as 0, input it into the blind zone length calculation model to calculate the initial blind zone length when the pose tilt angle is 0, and then calculate the distance difference between the current blind zone length and the initial blind zone length.

[0130] In one embodiment, the initial blind zone length may not be calculated using a blind zone length calculation model. When the cleaning robot confirms the installation tilt angle of its camera before leaving the factory, it can simultaneously store the pre-measured blind zone length required to calculate the installation tilt angle and set it as the initial blind zone length. When calculating the distance difference in this step, only the pose tilt angle or total tilt angle is input to the blind zone length calculation model, and after obtaining the current blind zone length output by the model, the distance difference is directly calculated based on the stored initial blind zone length.

[0131] S260: Based on the target image data and the preset virtual grid data, determine the target pixel coordinates of the target object in the virtual grid;

[0132] In this step, the cleaning robot confirms the pixel position (pixel coordinates) of the target object in the image based on the target image data, and determines the position of the target object in the virtual grid based on a pre-set virtual grid, which is represented by the target pixel coordinates. The specific details are similar to those in steps S222-S223 above, where the pixel coordinates of the pixel point in the virtual grid are determined; please refer to the relevant content for details, which will not be repeated here.

[0133] S270: Based on the distance difference, target pixel coordinates, and a preset distance matrix table, determine the actual detection distance between the target object and the cleaning robot.

[0134] In this step, the cleaning robot can initially determine the distance between the target object and the cleaning robot based on a preset distance matrix table and the determined target pixel coordinates. This initially determined distance corresponds to the distance when the camera's tilt angle is 0 degrees, and deviates somewhat from the actual distance. This deviation is mainly caused by the change in the length of the corresponding blind zone in the captured image due to the camera's tilt angle. Therefore, based on the distance difference corresponding to the tilt angle, the cleaning robot can effectively calibrate the initially determined distance or the preset distance matrix table, and then output the calibrated actual detection distance to improve the accuracy of distance detection.

[0135] This step can be implemented in two ways. First, the cleaning robot calibrates the detection distance corresponding to each pixel coordinate (pixel point) in a pre-defined distance matrix table based on the distance difference, obtaining a calibrated distance matrix table. This calibration process primarily corrects the vertical detection distance corresponding to each pixel point. Specifically, for the distance matrix table generated when the cleaning robot is in a horizontal pose, the vertical detection distances corresponding to all pixels are accumulated with the distance difference ΔCDis to generate a distance matrix table for the corresponding pitch angle. Then, the cleaning robot determines the actual detection distance between the target object and the cleaning robot based on the target pixel coordinates and the calibrated distance matrix table. The actual detection distance includes both the horizontal and vertical actual detection distances.

[0136] Secondly, the cleaning robot determines the initial detection distance between the target object and the cleaning robot based on the detection distances corresponding to each pixel coordinate in the preset distance matrix table. The initial detection distance includes the horizontal detection distance and the vertical detection distance. Then, the cleaning robot calculates the actual detection distance between the target object and the cleaning robot based on the distance difference and the initial detection distance. This process is the vertical detection distance corresponding to the target pixel coordinates, which is then added to the distance difference ΔCDis to obtain the calibrated vertical detection distance, while the horizontal detection distance remains unchanged.

[0137] In one embodiment, the actual detection distance of the target object relative to the camera and the target object's orientation relative to the camera can be represented by actual coordinates, where the origin of the actual coordinates represents the camera's position. A negative X-axis coordinate indicates the target object is to the left of the camera; conversely, a positive X-axis coordinate indicates it is to the right. The actual detection distance and orientation of the target object relative to the camera can also be stored as text or code in the relevant data of a distance matrix table for direct retrieval.

[0138] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating distance measurement based on the coordinates of the lower edge of a target object 3 in an image, as provided in an embodiment of this application. Figure 10 As shown, the cleaning robot uses a deep learning model to detect objects in RGB images, obtaining the minimum bounding rectangle of the corresponding object. Based on the nearest neighbor principle, the robot's processor selects the coordinates of the lower edge of the target object and performs a distance lookup and calibration based on a preset distance matrix to output the corresponding actual detection distance. For example, if the actual target object is placed 6cm horizontally to the left and 27cm vertically in front of the camera, the cleaning robot calculates the following distance using the above ranging method: -6.1cm horizontally and 27.2cm vertically. The error is less than 1cm, indicating that the actual detection distance output by the cleaning robot's virtual grid-based ranging method has high accuracy.

[0139] This application provides a virtual mesh-based ranging method and a cleaning robot. It acquires the current pose of the cleaning robot and relevant data from the image in front of the robot using a gyroscope and camera. Combining this with a pre-set distance matrix table and a blind zone length calculation model, it outputs a calibration deviation value, thereby calibrating the detection distance between the target object in front and the cleaning robot. This application is applicable to planar ranging and is easy to use. It effectively improves the distance detection accuracy of the cleaning robot while significantly reducing manufacturing costs. It requires minimal resources and offers high precision. Furthermore, it enhances the obstacle avoidance performance of the cleaning robot, making its operation safer and more efficient, meeting users' actual needs, and contributing to improved robot lifespan and user experience.

[0140] This application provides a computer-readable storage medium storing a computer program. The computer program can be executed by a processor 11 to perform a ranging method based on a virtual grid.

[0141] If a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0142] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A ranging method based on a virtual grid, characterized in that, The method includes: Acquire the tilt angle of the cleaning robot in its current pose state and the target image data collected for the target object; Based on the preset blind zone length calculation model and the pose tilt angle, the current blind zone length is calculated, and the distance difference between the current blind zone length and the known initial blind zone length is obtained. Based on the target image data and the preset virtual grid data, the target pixel coordinates of the target object in the virtual grid are determined; Based on the distance difference, the target pixel coordinates, and a preset distance matrix table, the actual detection distance between the target object and the cleaning robot is determined. Before calculating the current blind zone length based on the preset blind zone length calculation model and the pose tilt angle, the blind zone length calculation model is obtained by the following method: acquiring multiple sets of test data, including the camera installation height, the cleaning robot pose tilt angle and the corresponding blind zone length; fitting the numerical relationship curve of the installation height, the pose tilt angle and the corresponding blind zone length based on the multiple sets of test data, and establishing the blind zone length calculation model.

2. The ranging method based on a virtual grid according to claim 1, characterized in that, Determining the actual detection distance between the target object and the cleaning robot based on the distance difference, the target pixel coordinates, and a preset distance matrix table includes: Based on the distance difference, the detection distance corresponding to each pixel coordinate in the preset distance matrix table is calibrated; Based on the target pixel coordinates and the calibrated distance matrix, the actual detection distance between the target object and the cleaning robot is determined.

3. The ranging method based on a virtual grid according to claim 1, characterized in that, Determining the actual detection distance between the target object and the cleaning robot based on the distance difference, the target pixel coordinates, and a preset distance matrix table includes: Based on the target pixel coordinates and the detection distances corresponding to each pixel coordinate in the preset distance matrix table, the initial detection distance between the target object and the cleaning robot is determined; Based on the distance difference and the initial detection distance, the actual detection distance between the target object and the cleaning robot is calculated.

4. The ranging method based on a virtual grid according to claim 1, characterized in that, The calculation of the current blind zone length based on a preset blind zone length calculation model and the pose tilt angle, and the obtaining of the distance difference between the current blind zone length and the known initial blind zone length, includes: Based on the pose tilt angle and the camera mounting tilt angle, calculate the total tilt angle of the camera; The total tilt angle is input into the preset blind spot length calculation model to obtain the current blind spot length; Calculate the distance difference between the current blind zone length and the known initial blind zone length.

5. The ranging method based on a virtual grid according to claim 4, characterized in that, Before calculating the distance difference between the current blind zone length and the known initial blind zone length, the method further includes: The installation tilt angle is input into the preset blind zone length calculation model to obtain the known initial blind zone length.

6. The ranging method based on a virtual grid according to claim 1, characterized in that, Before determining the target pixel coordinates of the target object in the virtual grid based on the target image data and preset virtual grid data, the method further includes: The installation tilt angle of the camera is calculated based on the camera's installation height, the preset vertical field of view, and the length of the detected blind zone. Based on the pixel coordinates of each intersection point in the virtual grid and the detection distance corresponding to each intersection point, the detection distance corresponding to all pixels in the virtual grid at the installation tilt angle is determined and stored as a distance matrix table.

7. The ranging method based on a virtual grid according to claim 6, characterized in that, The determination of the detection distance for all pixels in the virtual mesh at the specified installation tilt angle, based on the pixel coordinates of each intersection point and the corresponding detection distance, includes: Based on the pixel coordinates of each pixel in the virtual grid and its adjacent intersection points, calculate the actual relative distance between the pixel and the adjacent intersection points; Based on the detection distances corresponding to the adjacent intersections and the actual relative distances, the detection distances corresponding to all pixels in the virtual grid are calculated.

8. The ranging method based on a virtual grid according to claim 1, characterized in that, Before determining the target pixel coordinates of the target object in the virtual grid based on the target image data and preset virtual grid data, the method further includes: Adjust the camera's installation tilt angle based on a preset unit angle value; When the tilt angle of the cleaning robot is 0, determine the detection distance of all pixels in the virtual grid under different installation tilt angles.

9. A cleaning robot, characterized in that, The cleaning robot includes: A gyroscope is used to detect the tilt angle of the cleaning robot. At least one camera is used to capture images of the target; A processor, connected to the gyroscope and the camera, is configured to execute the virtual grid-based ranging method according to any one of claims 1-8.

Images