Real-time visual obstacle avoidance method, device and computer equipment for mobile robot

Abstract

The application relates to a real-time visual obstacle avoidance method and device for a mobile robot and a computer device, and particularly relates to the technical field of robots. The method comprises the following steps: projecting current frame point cloud data to an XY two-dimensional plane under a robot body coordinate system; dividing the XY two-dimensional plane into multiple regions, and calculating a point cloud distance value corresponding to each region, wherein the point cloud distance value is used for representing the distance between point cloud data in the region and the mobile robot; and correcting the speed of the mobile robot based on the point cloud distance values corresponding to the multiple regions to avoid obstacles. Based on the technical solution, the calculation amount required by the mobile robot for real-time visual obstacle avoidance can be reduced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of robots, in particular to a real-time visual obstacle avoidance method and device for a mobile robot and a computer device. BACKGROUND

[0002] Obstacle avoidance is one of the core functions of a robot, which enables the robot to successfully complete a moving task while avoiding harm to itself and others or objects.

[0003] In the related art, obstacle avoidance is dependent on a map maintained by the robot, which can be a global map or a local map. The robot plans a moving trajectory based on the maintained map to achieve obstacle avoidance during movement.

[0004] Based on the above related art, obstacle avoidance requires a large amount of calculation due to the need to maintain a map. SUMMARY

[0005] The present application provides a real-time visual obstacle avoidance method and device for a mobile robot and a computer device, which technical solution is as follows.

[0006] In one aspect, a real-time visual obstacle avoidance method for a mobile robot is provided, which comprises:

[0007] projecting current frame point cloud data to an XY two-dimensional plane under a robot body coordinate system;

[0008] dividing the XY two-dimensional plane into multiple regions and calculating a point cloud distance value corresponding to each region, the point cloud distance value being used to represent the distance between the point cloud data in the region and the mobile robot;

[0009] correcting the speed of the mobile robot based on the point cloud distance values corresponding to the multiple regions to perform obstacle avoidance.

[0010] In another aspect, a real-time visual obstacle avoidance device for a mobile robot is provided, which comprises:

[0011] a two-dimensional plane projection module configured to project current frame point cloud data to an XY two-dimensional plane under a robot body coordinate system;

[0012] a region division module configured to divide the XY two-dimensional plane into multiple regions and calculate a point cloud distance value corresponding to each region, the point cloud distance value being used to represent the distance between the point cloud data in the region and the mobile robot;

[0013] a speed correction module configured to correct the speed of the mobile robot based on the point cloud distance values corresponding to the multiple regions to perform obstacle avoidance.

[0014] In yet another aspect, a computer-readable storage medium is provided, the storage medium having stored therein at least one instruction, the at least one instruction being loadable into a memory of a processor and executable by the processor to implement the real-time visual obstacle avoidance method of the mobile robot described above.

[0015] In yet another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the real-time visual obstacle avoidance method of the mobile robot described above.

[0016] The technical solutions provided in the present application can include the following beneficial effects:

[0017] The current frame point cloud data is projected onto an XY two-dimensional plane to complete preprocessing of the point cloud data, then the XY two-dimensional plane is divided into regions of a certain granularity, the point cloud distance values corresponding to each region are obtained, and then obstacle avoidance decisions are made according to these values to correct the speed of the mobile robot. On the one hand, this method does not need to use multiple frames of point cloud data to maintain a map, but only needs to process the current frame of point cloud data. On the other hand, the current frame of point cloud data is projected onto an XY two-dimensional plane, and the point cloud distance values required for subsequent obstacle avoidance decisions are only related to the values of the current frame of point cloud data on the XY two-dimensional plane, greatly reducing the amount of calculation required for real-time visual obstacle avoidance of the mobile robot. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is a method flowchart of the real-time visual obstacle avoidance method of the mobile robot according to an exemplary embodiment.

[0020] Figure 2 is a method flowchart of the real-time visual obstacle avoidance method of the mobile robot according to an exemplary embodiment.

[0021] Figure 3 is a schematic diagram of two-dimensional plane segmentation according to an exemplary embodiment.

[0022] Figure 4 is a method flowchart of the real-time visual obstacle avoidance method of the mobile robot according to an exemplary embodiment.

[0023] Figure 5 is a schematic diagram of the association between different coordinate systems according to an example embodiment.

[0024] Figure 6 is a method flow chart of a real-time visual obstacle avoidance method of a mobile robot according to an example embodiment.

[0025] Figure 7 is a structural block diagram of a real-time visual obstacle avoidance device of a mobile robot according to an example embodiment.

[0026] Figure 8 is a schematic diagram of a computer device provided according to an example embodiment. DETAILED DESCRIPTION

[0027] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0028] It should be understood that the "indication" mentioned in the embodiments of the present application can be direct indication, indirect indication, or can be an indication of an associated relationship. For example, A indicates B, which can mean that B can be obtained through A; or it can mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; or it can mean that A and B have an associated relationship.

[0029] In the description of the embodiments of the present application, the term "corresponding" can mean that there is a direct or indirect corresponding relationship between the two, or it can mean that there is an associated relationship between the two, or it can mean an indication and being indicated, configuration and being configured, etc.

[0030] In the embodiments of the present application, "predefined" can be realized by pre-saving corresponding codes, tables or other means for indicating related information in devices (such as terminal devices and network devices), and the present application does not limit the specific implementation manner.

[0031] Obstacle avoidance is one of the core functions of a mobile robot, which enables the robot to successfully complete the mobile task while avoiding damage to itself and others or objects.

[0032] Current obstacle avoidance techniques mainly rely on the map maintained by the robot. One is a global cost map approach. This approach is based on the idea that obstacle avoidance is dependent on navigation, and since navigation relies on a global map, the robot can and should make full use of the information in the global map to better avoid obstacles. However, obstacle avoidance does not necessarily rely on a global map. In some tasks, such as visual following or Ultra Wide Band (UWB) following tasks, the robot does not have a global map to use. An alternative approach is to use a local map. The robot uses continuous sensor data to maintain a local map centered on the robot's position. The main problem with this approach is that maintaining the local map requires a large amount of computation, and this computation is redundant. For example, when the robot is mainly moving forward, the map behind the robot is actually useless. The robot should be concerned about the obstacle situation near the forward direction.

[0033] To solve the above problems, in the embodiments of the present application, a real-time obstacle avoidance method based on a camera is implemented. This method does not require maintaining a map. Instead, the current frame point cloud data collected by the camera is effectively and fully processed to remove points in the point cloud data that will not affect the movement of the robot. Then the remaining point cloud data is coordinate transformed and projected onto an XY two-dimensional plane to complete the preprocessing of the point cloud data. Then the XY two-dimensional plane is divided into a certain granularity of sector regions to obtain the nearest distance of the point cloud data of the obstacle in each sector region from the robot. Then the original speed is corrected according to these distances to complete the obstacle avoidance decision.

[0034] It should be noted that the real-time visual obstacle avoidance method for mobile robots provided by the embodiments of the present application can be partially executed by the mobile robot and partially executed by the processing device. The processing device can be a server, a terminal device (such as a smart phone, a tablet computer, a desktop computer, a notebook computer, etc.), or the like. However, it can also be entirely executed by the mobile robot or entirely executed by the processing device. The present application is not limited in terms of execution subject, as long as the steps disclosed in the embodiments of the present application are executed.

[0035] Next, the real-time visual obstacle avoidance method for mobile robots provided by the embodiments of the present application will be further described.

[0036] Figure 1 is a method flowchart of the real-time visual obstacle avoidance method for mobile robots according to an exemplary embodiment. As shown in Figure 1 The real-time visual obstacle avoidance method for mobile robots can include the following steps:

[0037] Step 110: Project the current frame point cloud data onto the XY two-dimensional plane under the robot body coordinate system.

[0038] The current frame point cloud data is real-time acquired by the mobile robot. For example, the mobile robot acquires the current frame point cloud data by using an external sensor such as a camera to collect point cloud data in real time.

[0039] The robot body coordinate system is a rectangular coordinate system with the mobile robot as the reference. The robot body coordinate system generally corresponds to the directions of the X-axis, the Y-axis, and the Z-axis, wherein the directions of the X-axis and the Y-axis form an XY two-dimensional plane.

[0040] The current frame point cloud data generally contains information in three dimensions. In this embodiment, after the current frame point cloud data is acquired, the current frame point cloud data is projected onto the XY two-dimensional plane of the robot body coordinate system, and only the information of the current frame point cloud data on the X-axis and the Y-axis of the robot body coordinate system is used subsequently.

[0041] Step 120: The XY two-dimensional plane is divided into multiple regions, and the point cloud distance value corresponding to each region is calculated. The point cloud distance value is used to represent the distance between the point cloud data in the region and the mobile robot.

[0042] In this embodiment, in order to study the distance between the current frame point cloud data and the mobile robot, the XY two-dimensional plane is divided into multiple two-dimensional regions, and the point cloud distance value corresponding to each region is calculated.

[0043] It can be understood that the shape and area of the multiple regions are not limited in this embodiment. The shape of each region can be the same or different, and the area of each region can be the same or different. The multiple regions can be combined into the XY two-dimensional plane.

[0044] It can be understood that each region corresponds to a point cloud distance value. The point cloud distance value corresponding to a region is a distance value obtained after studying the distance between each point cloud data in the region and the mobile robot in the XY two-dimensional plane.

[0045] For example, the XY two-dimensional plane is divided into n regions, and the point cloud distance values corresponding to the n regions are D[1], D[2], …, D[n], respectively.

[0046] Step 130: Based on the point cloud distance values corresponding to the multiple regions, the speed of the mobile robot is corrected to avoid obstacles.

[0047] In this embodiment, after the point cloud distance values corresponding to the multiple regions are calculated, obstacle avoidance decisions are made according to these values, and the speed of the mobile robot is corrected.

[0048] It is understandable that the correction of the speed of the mobile robot includes, but is not limited to: correction of the speed direction of the mobile robot and correction of the speed magnitude of the mobile robot.

[0049] For example, through information preprocessing and obstacle avoidance decision-making, the corrected speed of the mobile robot is determined, and then obstacle avoidance is executed. The motion control algorithm enables the actual movement of the mobile robot to track the corrected motion instructions, so that the mobile robot moves at the corrected speed.

[0050] In summary, the real-time visual obstacle avoidance method for a mobile robot provided in this embodiment projects the current frame's point cloud data onto the XY two-dimensional plane to complete point cloud data preprocessing. The XY two-dimensional plane is then divided into regions of a certain granularity. After obtaining the point cloud distance values ​​corresponding to each region, obstacle avoidance decisions are made based on these values, and the mobile robot's speed is corrected. On the one hand, this method eliminates the need to use multiple frames of point cloud data to maintain a map, requiring only the processing of the current frame's point cloud data. On the other hand, since the current frame's point cloud data is projected onto the XY two-dimensional plane, the point cloud distance values ​​required for subsequent obstacle avoidance decisions are solely related to the values ​​of the current frame's point cloud data on the XY two-dimensional plane, significantly reducing the computational effort required for the mobile robot's real-time visual obstacle avoidance.

[0051] In an exemplary embodiment, the XY two-dimensional plane is divided into sector-shaped areas with a certain granularity, and then the point cloud distance value corresponding to each sector-shaped area is calculated.

[0052] Figure 2 FIG. 1 is a flow chart of a method for real-time visual obstacle avoidance of a mobile robot according to an exemplary embodiment. Figure 2 As shown, the real-time visual obstacle avoidance method of the mobile robot may include the following steps:

[0053] Step 210: Project the current frame point cloud data onto the XY two-dimensional plane in the robot body coordinate system.

[0054] The specific implementation of this step can be found in the above embodiment and will not be repeated here.

[0055] Step 220: Divide the XY two-dimensional plane into a plurality of sector-shaped areas according to the field of view angle.

[0056] The field of view angle is the field of view range corresponding to the current frame point cloud data. In this embodiment, the XY two-dimensional plane is divided by the field of view angle, thereby dividing the XY two-dimensional plane into multiple sector-shaped areas.

[0057] Exemplary, with reference to Figure 3 , the field of view angle is ψ, and the XY two-dimensional plane is divided into n sectors. Then for the i-th point cloud data in a frame of point cloud data, the corresponding sector area p can be foundi :

[0058]

[0059] wherein, denotes a floor symbol, and the ceiling operation can also be used instead, X R , Y R denotes the X-axis, Y-axis coordinate values of the point cloud data in the robot body coordinate system.

[0060] Step 230: Calculate the distance values between all point cloud data in each sector region and the mobile robot, to form a distance value set corresponding to each sector region.

[0061] For example, the distance value between the ith point cloud data and the mobile robot can be calculated according to the following formula:

[0062]

[0063] Step 240: Take the minimum value or the second minimum value in the distance value set corresponding to each sector region as the point cloud distance value corresponding to each sector region.

[0064] In this embodiment, the minimum value or the second minimum value of the distance values between the point cloud data in a sector region and the mobile robot is taken as the point cloud distance value corresponding to the sector region. The point cloud distance values corresponding to multiple sector regions can form a one-dimensional array.

[0065] It can be understood that there may be outliers in the point cloud data. In the case where the second minimum value of the distance values between the point cloud data in a sector region and the mobile robot is taken as the point cloud distance value corresponding to the sector region, the influence of outliers can be effectively removed.

[0066] For example, if the point cloud distance value corresponding to a sector region is considered to be the minimum value in the distance value set corresponding to the sector region, the point cloud distance value D[j] corresponding to the jth sector region can be calculated according to the following formula:

[0067]

[0068] Step 250: Based on the point cloud distance value corresponding to the central sector region in the multiple sector regions, correct the forward speed of the mobile robot, and the size of the forward speed is proportional to the size of the point cloud distance value corresponding to the central sector region.

[0069] The central sector region refers to a sector region in the middle of the plurality of sector regions. The central sector region can be understood as a sector region in the forward direction of the mobile robot, and accordingly, the point cloud distance value corresponding to the central sector region can be considered as the shortest distance between the mobile robot and the obstacle in the forward direction. Therefore, the forward speed of the mobile robot can be corrected by the point cloud distance value corresponding to the central sector region.

[0070] For example, the smaller the point cloud distance value corresponding to the central sector region, the smaller the forward speed; the larger the point cloud distance value corresponding to the central sector region, the larger the forward speed.

[0071] Step 260: correcting the rotation speed or side shift speed of the mobile robot based on the distance difference between the point cloud distance value corresponding to the adjacent sector region of the central sector region and the point cloud distance value corresponding to the central sector region, the rotation speed or side shift speed being proportional to the size of the distance difference.

[0072] The distance difference between the point cloud distance values corresponding to the adjacent sector region and the central sector region can be considered as a distance difference obtained according to the shortest distance between the mobile robot and the obstacle in the side direction and the shortest distance between the mobile robot and the obstacle in the forward direction. Therefore, the rotation speed or side shift speed of the mobile robot can be corrected by the distance difference.

[0073] For example, in the case where the point cloud distance value is positive, the larger the point cloud distance value, the larger the rotation speed or side shift speed; in the case where the point cloud distance value is negative, the rotation speed or side shift speed can be 0.

[0074] In summary, the real-time visual obstacle avoidance method of the mobile robot provided in the embodiment divides the XY two-dimensional plane into sector regions of certain granularity according to the field of view angle, and then calculates the point cloud distance value corresponding to each sector region, compresses a large amount of data into a one-dimensional array composed of the point cloud distance values corresponding to each sector region, and can better study the distance between the current frame point cloud data and the mobile robot in different angular directions.

[0075] In addition, when determining the point cloud distance value corresponding to the sector region, the second smallest distance value between the point cloud data in the sector region and the mobile robot can be taken as the point cloud distance value corresponding to the sector region, thereby effectively removing the influence of outliers.

[0076] In addition, according to the point cloud distance value corresponding to each sector region, the forward speed, rotation speed or side shift speed of the mobile robot can be accurately corrected, thereby achieving the obstacle avoidance effect through the correction of the speed.

[0077] In the illustrative embodiment, the current frame point cloud data is acquired by the camera, and filtering is performed on the point cloud data.

[0078] Figure 4 is a method flow chart of the real-time visual obstacle avoidance method of the mobile robot according to an exemplary embodiment. As shown in Figure 4 the real-time visual obstacle avoidance method of the mobile robot can include the following steps:

[0079] Step 410: filtering invalid point cloud data in the current frame point cloud data to obtain filtered current frame point cloud data, the invalid point cloud data including point cloud data higher than the camera and point cloud data near the ground.

[0080] The invalid point cloud data refers to point cloud data without reference for the obstacle avoidance of the mobile robot. In the embodiment, after the current frame point cloud data is acquired, the point cloud data higher than the camera and the point cloud data near the ground are identified as invalid point cloud data, and the invalid point cloud data in the current frame point cloud data is filtered.

[0081] It can be understood that the camera is generally arranged at the upper part of the mobile robot, the point cloud data higher than the camera corresponds to an obstacle above the mobile robot, which will not affect the advance of the mobile robot, and thus the point cloud data higher than the camera is identified as invalid point cloud data; in addition, the mobile robot generally has a certain obstacle crossing ability for low obstacles, the point cloud data near the ground corresponds to a low obstacle, which will not affect the advance of the mobile robot, and thus the point cloud data near the ground is identified as invalid point cloud data.

[0082] In a possible implementation, step 410 can be replaced by:

[0083] (1) acquiring, by the camera, a representation of the current frame point cloud data in a pixel coordinate system

[0084] (2) performing coordinate change calculation on the representation of the current frame point cloud data in the pixel coordinate system to obtain a representation of the current frame point cloud data in a camera body coordinate system and

[0085]

[0086] wherein, is a camera intrinsic parameter matrix.

[0087] That is,

[0088] (3) considering point cloud data satisfying as point cloud data higher than the camera, and considering point cloud data satisfying as point cloud data near the ground.The point cloud data is considered to be point cloud data near the ground.

[0089] wherein, θ is the angle between the Z axis of the camera coordinate system and the X axis of the robot body coordinate system, h is the distance between the camera and the bottom of the mobile robot, and δ is the distance threshold.

[0090] In the embodiment, the mobile robot carries a camera, which is a depth camera, and the camera has no change in roll angle. A rectangular coordinate system with the mobile robot as the reference is the robot body coordinate system, and a rectangular coordinate system with the camera as the reference is the camera body coordinate system.

[0091] As shown in Figure 5 , the origin of the robot body coordinate system and the origin of the camera coordinate system can be fixedly connected, the X axis (x in the figure C ) of the camera coordinate system is parallel to the Y axis (y in the figure R ) of the robot body coordinate system, and the angle between the Z axis (z in the figure C ) of the camera coordinate system and the X axis (x in the figure R ) of the robot body coordinate system is θ.

[0092] In the embodiment, the angle between the straight line formed between the point (0, Yc, Zc) and the origin of the coordinate system and the Z axis of the camera coordinate system can be considered, and therefore, when the above value is greater than θ, the point cloud data is point cloud data higher than the camera; the distance from the point cloud data to the ground in the robot body coordinate system can be considered, and therefore, when the above value is less than the distance threshold δ, the point cloud data is point cloud data near the ground.

[0093] (4) filtering the point cloud data higher than the camera and the point cloud data near the ground to obtain filtered current frame point cloud data.

[0094] Step 420: projecting the filtered current frame point cloud data to the XY two-dimensional plane in the robot body coordinate system.

[0095] In the embodiment, after the filtering of the invalid point cloud data in the current frame point cloud data is completed, the projection of the remaining point cloud data to the XY two-dimensional plane in the robot body coordinate system is performed.

[0096] In a possible implementation, step 420 can be replaced by:

[0097] representing the X axis data and Y axis data of the filtered current frame point cloud data in the camera body coordinate system The coordinate change calculation is performed to obtain X-axis data and Y-axis data of the filtered current frame point cloud data in the robot body coordinate system and

[0098]

[0099] wherein θ is an included angle between the Z-axis of the camera coordinate system and the X-axis of the robot body coordinate system.

[0100] That is,

[0101] In a possible implementation, the filtered current frame point cloud data is first down-sampled, and then projected to the XY two-dimensional plane in the robot body coordinate system.

[0102] It can be understood that, in the above steps, the posture of the camera is described as a change without a roll angle, that is, the X-axis of the camera coordinate system is parallel to the Y-axis of the robot body coordinate system; or it can be assumed that the change is without a pitch angle, that is, the X-axis of the camera body coordinate system is parallel to the Z-axis of the robot body coordinate system, and the processing manner under this assumption is similar to that in the above steps, which will not be described herein again.

[0103] Step 430: The XY two-dimensional plane is divided into a plurality of regions, and a point cloud distance value corresponding to each region is calculated, the point cloud distance value being used to represent the distance between the point cloud data in the region and the mobile robot.

[0104] The specific implementation manner of this step can be referred to the above embodiments, which will not be described herein again.

[0105] Step 440: Based on the point cloud distance values corresponding to the plurality of regions, the speed of the mobile robot is corrected to avoid obstacles.

[0106] The specific implementation manner of this step can be referred to the above embodiments, which will not be described herein again.

[0107] In summary, the real-time visual obstacle avoidance method of the mobile robot provided in this embodiment, after the current frame point cloud data is obtained, first identifies the point cloud data above the camera and the point cloud data near the ground as invalid point cloud data, and filters the invalid point cloud data in the current frame point cloud data, so that subsequent processing of the invalid point cloud data is not required, and the calculation amount is simplified.

[0108] In addition, when the coordinate transformation from the camera coordinate system to the robot body coordinate system is performed, the point cloud data does not need to be completely transformed, only the X-axis data and the Y-axis data need to be calculated, and the calculation amount is simplified.

[0109] In combination with reference Figure 6The workflow of the real-time visual obstacle avoidance method of the mobile robot shown in the above embodiments is described as follows:

[0110] Step 601: Start.

[0111] Step 602: Calculate camera parameters.

[0112] The camera parameters include the angle θ between the Z-axis of the camera coordinate system and the X-axis of the robot body coordinate system, and the distance h between the camera and the bottom of the mobile robot.

[0113] Step 603: Obtain a frame of point cloud data.

[0114] Step 604: Filter points above the top of the camera and near the ground.

[0115] The specific algorithm of the filtering can be referred to the above embodiments, which is not described herein again.

[0116] Step 605: Calculate the X R , Y R of the remaining points in the robot body coordinate system.

[0117] After the filtering of the point cloud data is completed, the remaining points are subjected to coordinate transformation to calculate the X R , Y R in the robot body coordinate system. The specific algorithm of the coordinate transformation can be referred to the above embodiments, which is not described herein again.

[0118] Step 606: Calculate the corresponding sector area.

[0119] For any point cloud data, the corresponding sector area can be calculated. The specific algorithm of calculating the sector area to which the point cloud data belongs can be referred to the above embodiments, which is not described herein again.

[0120] Step 607: Calculate the point cloud distance value corresponding to each sector area.

[0121] The point cloud distance value corresponding to each sector area can be the minimum value or the second minimum value in the distance value set corresponding to each sector area, and the distance value set corresponding to each sector area includes the distance values between all point cloud data in each sector area and the mobile robot.

[0122] Step 608: Correct the speed.

[0123] The forward speed, rotation speed or side shift speed of the mobile robot can be corrected according to the point cloud distance value corresponding to each sector area.

[0124] In addition, after step 608 is completed, jump to step 603 to perform again.

[0125] It should be noted that the above method embodiments can be implemented alone or in combination, and the present application does not limit this.

[0126] Figure 7 is a structure block diagram of a real-time visual obstacle avoidance device of a mobile robot according to an exemplary embodiment. The device comprises:

[0127] A two-dimensional plane projection module 701 is configured to project current frame point cloud data to an XY two-dimensional plane under a robot body coordinate system.

[0128] A region segmentation module 702 is configured to segment the XY two-dimensional plane into multiple regions and calculate a point cloud distance value corresponding to each region, the point cloud distance value being used to represent a distance between point cloud data in the region and the mobile robot.

[0129] A speed correction module 703 is configured to correct a speed of the mobile robot based on the point cloud distance values corresponding to the multiple regions for obstacle avoidance.

[0130] In a possible implementation, the region segmentation module 702 is further configured to:

[0131] According to a field of view angle, the XY two-dimensional plane is divided into multiple sector regions.

[0132] A distance value between all point cloud data in each sector region and the mobile robot is calculated to form a distance value set corresponding to each sector region.

[0133] A minimum value or a second minimum value in the distance value set corresponding to each sector region is taken as a point cloud distance value corresponding to each sector region.

[0134] In a possible implementation, the speed correction module 703 is further configured to:

[0135] Based on a point cloud distance value corresponding to a central sector region in the multiple sector regions, a forward speed of the mobile robot is corrected, the forward speed being directly proportional to a size of the point cloud distance value.

[0136] In a possible implementation, the speed correction module 703 is further configured to:

[0137] Based on a distance difference between the point cloud distance values corresponding to the central sector region and an adjacent sector region, a rotation speed or a lateral movement speed of the mobile robot is corrected, the rotation speed or the lateral movement speed being directly proportional to a size of the distance difference.

[0138] In a possible implementation, the two-dimensional plane projection module 701 is further configured to:

[0139] Filtering invalid point cloud data in the current frame point cloud data to obtain filtered point cloud data of the current frame;

[0140] Projecting the filtered point cloud data of the current frame onto the XY two-dimensional plane of the robot body coordinate system;

[0141] The invalid point cloud data includes: point cloud data above the camera and point cloud data near the ground.

[0142] In a possible implementation, the two-dimensional plane projection module 701 is further configured to:

[0143] Through camera acquisition, obtain the representation of the current frame point cloud data in the pixel coordinate system

[0144] Perform coordinate change calculation on the representation of the current frame point cloud data in the pixel coordinate system to obtain the representation of the current frame point cloud data in the camera body coordinate system and

[0145]

[0146] will satisfy The point cloud data is considered to be higher than the point cloud data of the camera, satisfying The point cloud data is considered to be point cloud data near the ground;

[0147] Filtering the point cloud data above the camera and the point cloud data near the ground to obtain filtered point cloud data of the current frame;

[0148] Among them, the X axis of the camera coordinate system is parallel to the Y axis of the robot body coordinate system. is the camera intrinsic parameter matrix, θ is the angle between the Z axis of the camera coordinate system and the X axis of the robot body coordinate system, h is the distance between the camera and the bottom of the mobile robot, and δ is the distance threshold.

[0149] In a possible implementation, the two-dimensional plane projection module 701 is further configured to:

[0150] Representation of the X-axis data and Y-axis data in the filtered current frame point cloud data in the camera body coordinate system Perform coordinate change calculation to obtain the representation of the X-axis data and Y-axis data in the filtered current frame point cloud data in the robot body coordinate system and

[0151] Where θ is the angle between the Z axis of the camera coordinate system and the X axis of the robot body coordinate system.

[0152] In a possible implementation, the current frame point cloud data corresponds to a forward direction of the mobile robot.

[0153] It should be noted that the real-time visual obstacle avoidance device of the mobile robot provided in the above examples is only exemplified by the division of the above functional modules. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above examples belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0154] Please refer to Figure 8 which is a schematic diagram of a computer device according to an exemplary embodiment of the present application. The computer device includes a memory and a processor. The memory is used to store a computer program. The computer program is executed by the processor to implement the real-time visual obstacle avoidance method of the mobile robot described above.

[0155] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0156] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the methods in the embodiments of the present disclosure. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the methods in the above method embodiments.

[0157] The memory can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required by at least one function, and the like, and the data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some embodiments, the memory can optionally include a memory that is remotely located with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0158] In an exemplary embodiment, a computer readable storage medium storing at least one computer program is also provided, the at least one computer program is loaded and executed by the processor to implement all or part of the steps of the above method. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0159] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed here. It is intended that the present application cover any and all variations of the application that come within the scope of the present application, along with all of the equivalents thereof. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0160] It should be understood that the application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should be limited only by the appended claims.

Claims

1. A real-time vision-based obstacle avoidance method for a mobile robot, characterized in that, The method comprises: projecting current frame point cloud data to an XY two-dimensional plane under a robot body coordinate system; dividing the XY two-dimensional plane into multiple regions and calculating point cloud distance values corresponding to each region, the point cloud distance values being used to represent distances between point cloud data in the regions and the mobile robot, comprising: dividing the XY two-dimensional plane into multiple sector regions according to a field of view angle; calculating distance values between all point cloud data in each sector region and the mobile robot to form a distance value set corresponding to each sector region; taking a minimum value or a second minimum value in the distance value set corresponding to each sector region as a point cloud distance value corresponding to each sector region; correcting a speed of the mobile robot for obstacle avoidance based on the point cloud distance values corresponding to the multiple regions, comprising: correcting a rotation speed or a side shift speed of the mobile robot based on a distance difference value between the point cloud distance values corresponding to a central sector region and adjacent sector regions, the rotation speed or the side shift speed being directly proportional to the size of the distance difference value.

2. The method of claim 1, wherein, The correcting of the speed of the mobile robot for obstacle avoidance based on the point cloud distance values corresponding to the multiple regions comprises: correcting a forward speed of the mobile robot based on the point cloud distance value corresponding to a central sector region in the multiple sector regions, the size of the forward speed being directly proportional to the size of the point cloud distance value.

3. The method of claim 1, wherein, The projecting of the current frame point cloud data to the XY two-dimensional plane under the robot body coordinate system comprises: filtering invalid point cloud data in the current frame point cloud data to obtain filtered current frame point cloud data; projecting the filtered current frame point cloud data to the XY two-dimensional plane under the robot body coordinate system; wherein the invalid point cloud data comprises point cloud data higher than a camera and point cloud data near a ground surface.

4. The method of claim 3, wherein, The filtering of the invalid point cloud data in the current frame point cloud data to obtain the filtered current frame point cloud data comprises: acquire the representation of the current frame point cloud data in the pixel coordinate system through camera acquisition perform coordinate change calculation on the representation of the current frame point cloud data in the pixel coordinate system to obtain the representation of the current frame point cloud data in the camera body coordinate system and Point cloud data satisfying is considered to be point cloud data above the camera, point cloud data satisfying is considered to be point cloud data near the ground; filtering the point cloud data higher than the camera and the point cloud data near the ground surface to obtain the filtered current frame point cloud data. wherein the X axis of the camera coordinate system is parallel to the Y axis of the robot body coordinate system, is an intrinsic parameter matrix of the camera, θ is an angle between the Z axis of the camera coordinate system and the X axis of the robot body coordinate system, h is a distance between the camera and the bottom of the mobile robot, and δ is a distance threshold.

5. The method of claim 3, wherein, The projecting of the filtered current frame point cloud data to the XY two-dimensional plane under the robot body coordinate system comprises: the representation of the X-axis data and the Y-axis data in the current frame of point cloud data after filtering in the camera body coordinate system perform coordinate change calculation to obtain the representation of the X-axis data and the Y-axis data in the current frame of point cloud data after filtering in the robot body coordinate system and wherein θ is an included angle between a Z axis of a camera coordinate system and an X axis of a robot body coordinate system.

6. The method of claim 1, wherein: the current frame point cloud data corresponds to a forward direction of the mobile robot.

7. A real-time visual obstacle avoidance device for a mobile robot, comprising: The device comprises: a two-dimensional plane projection module configured to project current frame point cloud data to an XY two-dimensional plane under a robot body coordinate system; a region division module configured to divide the XY two-dimensional plane into multiple regions and calculate point cloud distance values corresponding to each region, the point cloud distance values being used to represent distances between point cloud data in the regions and the mobile robot, comprising: dividing the XY two-dimensional plane into multiple sector regions according to a field of view angle; calculating distance values between all point cloud data in each sector region and the mobile robot to form a distance value set corresponding to each sector region; The minimum or second minimum value in the distance value set corresponding to each sector region is taken as the point cloud distance value corresponding to each sector region; The speed correction module is configured to correct the speed of the mobile robot based on the point cloud distance values corresponding to the plurality of regions to avoid obstacles, and includes: The rotational speed or lateral movement speed of the mobile robot is corrected based on the distance difference between the point cloud distance values corresponding to the central sector region and the adjacent sector region, and the rotational speed or lateral movement speed is proportional to the size of the distance difference.

8. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which is loaded and executed by the processor to implement the real-time visual obstacle avoidance method of the mobile robot as claimed in any one of claims 1 to 6.

Images