Robot fall environment determination method, apparatus, and device

Abstract

The application discloses a robot falling environment determination method, device and equipment, and belongs to the technical field of robots. The method comprises the following steps: determining falling point cloud data from regional point cloud data, layering the falling point cloud data in a vertical direction to obtain multi-layer layered point cloud data, then determining the road condition corresponding to each layer of the layered point cloud data according to each layer of the multi-layer layered point cloud data, and determining whether the environment in front of the robot is a falling environment according to the road conditions corresponding to the multi-layer layered point cloud data. In this way, the road conditions corresponding to the multi-layer layered point cloud data obtained after the falling point cloud is layered are determined, and whether the environment in front of the robot is a falling environment is determined according to different combinations of the road conditions corresponding to different layers of the layered point cloud data, so that the dependence on the correlation between all coordinate points included in the falling point cloud data is reduced, the accuracy of determining the falling environment is improved, and the probability of robot falling or false stopping is reduced.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method, apparatus, and device for determining the fall environment of a robot. Background Technology

[0002] As we all know, the smooth movement of a robot depends on its assessment of the environment ahead. The environment in which a robot could fall is called a fall environment. Accurately determining whether the environment ahead is a fall environment is fundamental to preventing falls. For example, if it is determined that there are downward steps in the environment ahead, then the environment is considered a fall environment, and the robot should be stopped to avoid a fall and ensure its safety.

[0003] In existing technologies, point cloud data of the area within the field of view of the camera mounted on the robot can be acquired first. Fall point cloud data can then be determined from this data. If the road condition corresponding to the fall point cloud data is determined to be a staircase, then the environment in front of the robot is identified as a fall environment. For example, a plane fitting can be performed on the fall point cloud data to obtain a fitted plane. If the tilt angle of the fitted plane is less than a threshold, then the road condition corresponding to the fall point cloud data is determined to be a staircase. The set of coordinate points in the area point cloud data is the set of three-dimensional coordinates of multiple pixels within the camera's field of view in the robot's coordinate system. The fall point cloud data includes the set of coordinate points in the area point cloud data whose vertical coordinates are less than the vertical coordinates of the robot's horizontal plane.

[0004] However, the aforementioned method for determining the fall environment based on fall point cloud data relies on a fitted plane determined from the fall point cloud data, i.e., it depends on the correlation between all coordinate points included in the fall point cloud data. This results in low accuracy in determining the fall environment and a high probability of robot falls or accidental stops. For example, errors in the three-dimensional coordinates of the set of coordinate points included in the fall point cloud data can cause errors in the tilt angle of the fitted plane. This can lead to incorrectly identifying a step in the road conditions corresponding to the fall point cloud data as a slope, causing the robot to fall, or incorrectly identifying a slope in the road conditions corresponding to the fall point cloud data as a step, causing the robot to stop accidentally. Summary of the Invention

[0005] This application provides a method, apparatus, and device for determining the robot's fall environment, which can accurately determine whether the environment ahead of the robot is a fall environment, reducing the probability of the robot falling or stopping unnecessarily. The technical solution is as follows:

[0006] Firstly, a method for determining the fall environment of a robot is provided, the method comprising:

[0007] The fall point cloud data is determined from the regional point cloud data. The regional point cloud data includes a set of coordinate points that are the set of three-dimensional coordinate points corresponding to multiple pixels in the field of view of the camera mounted on the robot in the robot coordinate system.

[0008] The drop point cloud data is layered in the vertical direction to obtain multi-layered point cloud data.

[0009] Based on each layer of the multi-layered point cloud data, determine the road conditions corresponding to each layer of the point cloud data.

[0010] Based on the road conditions corresponding to the multi-layered point cloud data, it is determined whether the environment in front of the robot is a fall environment.

[0011] As an example, determining the road conditions corresponding to each layer of the multi-layered point cloud data based on each layer of the multi-layered point cloud data includes:

[0012] For the first layer point cloud data in the multi-layer point cloud data, if the number of coordinate points included in the first layer point cloud data is greater than or equal to the first preset number, then the road condition corresponding to the first layer point cloud data is determined to be a valid road condition, and the first layer point cloud data is any layer point cloud data of the multi-layer point cloud data.

[0013] If the number of coordinate points included in the first layered point cloud data is less than the first preset number, then the road condition corresponding to the first layered point cloud data is determined to be an invalid road condition.

[0014] As an example, the step of determining the road condition corresponding to the first layered point cloud data as a valid road condition if the number of coordinate points included in the first layered point cloud data is greater than or equal to a first preset number includes:

[0015] If the number of coordinate points included in the first layered point cloud data is greater than or equal to the first preset number, then the first layered point cloud data is fitted with a plane to obtain the fitting plane of the first layered point cloud data.

[0016] Based on the first tilt angle of the fitting plane of the first layered point cloud data, the type of valid road condition to which the road condition corresponding to the first layered point cloud data belongs is determined. The first tilt angle is the angle between the normal vector of the fitting plane of the first layered point cloud data and the horizontal plane where the robot is located.

[0017] As an example, the types of valid road conditions include at least steps, dangerous slopes, and ordinary slopes;

[0018] The step of determining the type of valid road condition corresponding to the road condition in the first layered point cloud data based on the first tilt angle of the fitted plane of the first layered point cloud data includes:

[0019] If the first tilt angle is less than or equal to the first preset angle, then the road condition corresponding to the first layered point cloud data is determined to be the step;

[0020] If the first tilt angle is greater than or equal to the second preset angle, then the road condition corresponding to the first layered point cloud data is determined to be the dangerous slope, and the second preset angle is greater than the first preset angle.

[0021] If the first tilt angle is greater than the first preset angle and less than the second preset angle, then the road condition corresponding to the first layered point cloud data is determined to be the ordinary slope.

[0022] As an example, the road conditions corresponding to the multi-layered point cloud data include at least steps and dangerous slopes;

[0023] The step of determining whether the environment ahead of the robot is a fall environment based on the road conditions corresponding to the multi-layered point cloud data includes:

[0024] A first value is determined, which is used to indicate the number of layers in the multi-layered point cloud data where the road condition is the step.

[0025] If the first value is greater than or equal to the first quantity threshold, then the environment in front of the robot is determined to be the fall environment, and the first quantity threshold is greater than 1;

[0026] If the first value is equal to the second quantity threshold, then in the multi-layered point cloud data, if the height of the first step corresponding to the road condition of the step and the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to the road condition of the step satisfy the preset conditions, the environment in front of the robot is determined to be the fall environment, and the second quantity threshold is greater than 1 and less than the first quantity threshold.

[0027] If the first value is equal to 1, then if the number of layers of the layered point cloud data corresponding to the road condition of the dangerous slope in the multi-layered point cloud data is greater than or equal to the preset number of layers, the environment in front of the robot is determined to be the fall environment.

[0028] As an example, the road conditions corresponding to the multi-layered point cloud data also include invalid road conditions and ordinary slopes, and the method further includes:

[0029] If the first value is equal to 1, then in the case that other road conditions in the road conditions corresponding to the multi-layered point cloud data are invalid road conditions except for the road condition that is the step, the second step height of the road condition corresponding to the step in the multi-layered point cloud data is determined.

[0030] If the height of the second step is greater than or equal to the height threshold, then the environment in front of the robot is determined to be the fall environment.

[0031] If the height of the second step is less than the height threshold, the robot is controlled to rotate by a third preset angle. If, based on the point cloud data of the area within the field of view of the robot's camera after rotation, it is determined that the ordinary inclined plane does not exist in the environment in front of the robot after rotation, then the environment in front of the robot before rotation is determined to be the fall environment.

[0032] As an example, the drop point cloud data is layered vertically to obtain multi-layered point cloud data, including...

[0033] In the vertical direction, the drop point cloud data is divided into layers with a preset height as the unit to obtain the multi-layered point cloud data, and the height of each layer of point cloud data in the vertical direction is the preset height.

[0034] As an example, before layering the drop point cloud data in the vertical direction, the method further includes:

[0035] A second value is determined, which indicates the number of coordinate points in the drop point cloud data whose vertical coordinate values ​​are less than the maximum height coordinate value;

[0036] If the second value is less than the second preset quantity, then layerable point cloud data is determined from the drop point cloud data. The layerable point cloud data includes a set of coordinate points in the drop point cloud data whose vertical coordinate values ​​are greater than or equal to the maximum height coordinate value. The step of layering the drop point cloud data in the vertical direction to obtain multi-layered point cloud data includes: layering the layerable point cloud data in the vertical direction to obtain the multi-layered point cloud data.

[0037] If the second value is greater than or equal to the second preset quantity, then the environment in front of the robot is determined to be a fall environment.

[0038] Secondly, a robot fall environment determination device is provided, the device comprising:

[0039] The first determining module is used to determine the fall point cloud data from the regional point cloud data. The regional point cloud data includes a set of coordinate points that is a set of three-dimensional coordinates of multiple pixels in the field of view of the camera mounted on the robot in the robot coordinate system.

[0040] The layering module is used to layer the drop point cloud data in the vertical direction to obtain multi-layered point cloud data;

[0041] The second determining module is used to determine the road conditions corresponding to each layer of the multi-layered point cloud data based on each layer of the multi-layered point cloud data.

[0042] The third determining module is used to determine whether the environment in front of the robot is a fall environment based on the road conditions corresponding to the multi-layered point cloud data.

[0043] As an example, the second determining module is further configured to determine the road condition corresponding to the first layer point cloud data as a valid road condition if the number of coordinate points included in the first layer point cloud data in the multi-layer point cloud data is greater than or equal to a first preset number, and the first layer point cloud data is any layer point cloud data of the multi-layer point cloud data.

[0044] If the number of coordinate points included in the first layered point cloud data is less than the first preset number, then the road condition corresponding to the first layered point cloud data is determined to be an invalid road condition.

[0045] As an example, the second determining module is further configured to perform plane fitting on the first layered point cloud data if the number of coordinate points included in the first layered point cloud data is greater than or equal to the first preset number, so as to obtain the fitting plane of the first layered point cloud data.

[0046] Based on the first tilt angle of the fitting plane of the first layered point cloud data, the type of valid road condition to which the road condition corresponding to the first layered point cloud data belongs is determined. The first tilt angle is the angle between the normal vector of the fitting plane of the first layered point cloud data and the horizontal plane where the robot is located.

[0047] As an example, the types of valid road conditions include at least steps, dangerous slopes, and ordinary slopes;

[0048] The second determining module is further configured to determine the road condition corresponding to the first layered point cloud data as the steps if the first tilt angle is less than or equal to the first preset angle;

[0049] If the first tilt angle is greater than or equal to the second preset angle, then the road condition corresponding to the first layered point cloud data is determined to be the dangerous slope, and the second preset angle is greater than the first preset angle.

[0050] If the first tilt angle is greater than the first preset angle and less than the second preset angle, then the road condition corresponding to the first layered point cloud data is determined to be the ordinary slope.

[0051] As an example, the road conditions corresponding to the multi-layered point cloud data include at least steps and dangerous slopes;

[0052] The third determining module is further configured to determine a first value, which indicates the number of layers in the multi-layered point cloud data corresponding to the road condition of the step.

[0053] If the first value is greater than or equal to the first quantity threshold, then the environment in front of the robot is determined to be the fall environment, and the first quantity threshold is greater than 1;

[0054] If the first value is equal to the second quantity threshold, then in the multi-layered point cloud data, if the height of the first step corresponding to the road condition of the step and the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to the road condition of the step satisfy the preset conditions, the environment in front of the robot is determined to be the fall environment, and the second quantity threshold is greater than 1 and less than the first quantity threshold.

[0055] If the first value is equal to 1, then if the number of layers of the layered point cloud data corresponding to the road condition of the dangerous slope in the multi-layered point cloud data is greater than or equal to the preset number of layers, the environment in front of the robot is determined to be the fall environment.

[0056] As an example, the road conditions corresponding to the multi-layered point cloud data also include invalid road conditions and ordinary slopes;

[0057] The third determining module is further configured to determine the second step height of the road condition corresponding to the step in the multi-layered point cloud data if the first value is equal to 1, and if the other road conditions in the road conditions corresponding to the multi-layered point cloud data are invalid road conditions except for the road condition that is the step.

[0058] If the height of the second step is greater than or equal to the height threshold, then the environment in front of the robot is determined to be the fall environment.

[0059] If the height of the second step is less than the height threshold, the robot is controlled to rotate by a third preset angle. If, based on the point cloud data of the area within the field of view of the robot's camera after rotation, it is determined that the ordinary inclined plane does not exist in the environment in front of the robot after rotation, then the environment in front of the robot before rotation is determined to be the fall environment.

[0060] As an example, the layering module is also used to layer the drop point cloud data in the vertical direction in units of a preset height to obtain the multi-layered point cloud data, wherein the height of each layer of point cloud data in the vertical direction is the preset height.

[0061] As an example, the device further includes a fourth determining module, a fifth determining module, and a sixth determining module;

[0062] The fourth determining module is used to determine a second value, which indicates the number of coordinate points in the drop point cloud data whose vertical coordinate values ​​are less than the maximum height coordinate value.

[0063] The fifth determining module is used to determine layerable point cloud data from the drop point cloud data if the second value is less than the second preset quantity. The layerable point cloud data includes a set of coordinate points in the drop point cloud data whose vertical coordinate values ​​are greater than or equal to the maximum height coordinate value.

[0064] The layering module is also used to layer the layerable point cloud data in the vertical direction to obtain the multi-layered point cloud data.

[0065] The sixth determining module is used to determine that the environment in front of the robot is a fall environment if the second value is greater than or equal to the second preset quantity.

[0066] Thirdly, a computer device is provided, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described method for determining the robot fall environment.

[0067] The beneficial effects of the technical solutions provided in this application are:

[0068] In this embodiment, fall point cloud data is first determined from regional point cloud data. Then, the fall point cloud data is layered vertically to obtain multi-layered point cloud data. Next, based on each layer of the multi-layered point cloud data, the road conditions corresponding to each layer are determined. Based on the road conditions corresponding to the multi-layered point cloud data, it is determined whether the environment ahead of the robot is a fall environment. The regional point cloud data includes a set of coordinate points that represent the set of three-dimensional coordinates of multiple pixels within the field of view of the robot's camera in the robot's coordinate system. In this way, the road conditions corresponding to each layer of the multi-layered point cloud data obtained after layering the fall point cloud are determined. Based on different combinations of road conditions corresponding to different layers of fall point cloud data, it is determined whether the environment ahead of the robot is a fall environment. This reduces the dependence on the correlation between all coordinate points included in the fall point cloud data when determining the fall environment, improves the accuracy of determining the fall environment, and reduces the probability of the robot falling or stopping unnecessarily. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart of a method for determining a robot fall environment provided in an embodiment of this application;

[0071] Figure 2 This is a schematic diagram of a robot encountering a downward step, provided in an embodiment of this application;

[0072] Figure 3 This is a flowchart of another method for determining the robot fall environment provided in an embodiment of this application;

[0073] Figure 4 This is a schematic diagram of a multi-layered point cloud data where the number of layers corresponding to the road condition of steps is 1, provided in an embodiment of this application.

[0074] Figure 5 This is a schematic diagram of the structure of a robot fall environment determination device provided in an embodiment of this application;

[0075] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0077] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0078] Before providing a detailed explanation of the embodiments of this application, the application scenarios of these embodiments will be described first.

[0079] The robot fall environment determination method provided in this application can be applied to robots equipped with cameras. It can accurately determine whether the environment in front of the robot is a fall environment, reducing the probability of the robot falling or stopping unnecessarily. The environment in which the robot might fall can be referred to as the fall environment.

[0080] The robot fall environment determination method provided in the embodiments of this application will be explained in detail below.

[0081] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for determining a robot fall environment according to an embodiment of this application. This method can be applied to a robot equipped with a camera, or to a computer device connected to the robot. The computer device can be a terminal, server, or embedded device, and the terminal can be a desktop computer or tablet computer. The following explanation will use a robot equipped with a camera as an example to illustrate the method for determining the robot fall environment. Figure 1 As shown, the method includes the following steps:

[0082] Step 101: The robot determines the fall point cloud data from the regional point cloud data.

[0083] The regional point cloud data includes a set of coordinate points that are the set of three-dimensional coordinate points corresponding to multiple pixels in the field of view of the camera mounted on the robot in the robot coordinate system. The fall point cloud data includes a set of coordinate points in the regional point cloud data whose vertical coordinate values ​​are less than or equal to the allowable error coordinate values. The coordinate points included in the fall point cloud data are located below the horizontal plane where the robot is located.

[0084] The robot coordinate system can also be called the robot body coordinate system. The robot coordinate system is a pre-set three-dimensional coordinate system. The coordinate value of a point in the vertical direction in the regional point cloud data refers to the coordinate value of the point on the Z-axis of the robot coordinate system.

[0085] As an example, the origin of the robot coordinate system can be the center of the robot or the center of the robot's chassis. The X-axis of the robot coordinate system points directly in front of the robot, the Y-axis points to the left of the robot, and the Z-axis points upwards of the robot. Furthermore, in this embodiment, the X-axis of the robot coordinate system is parallel to the horizontal plane where the robot is located, and the Z-axis is perpendicular to the horizontal plane where the robot is located. The X-axis can be located above or flush with the horizontal plane where the robot is located; this embodiment does not limit the specific location of the X-axis.

[0086] The tolerance coordinate value is a pre-set value, referring to a coordinate value on the Z-axis of the robot's coordinate system. The tolerance coordinate value is less than 0. For example, please refer to... Figure 2 , Figure 2 This is a schematic diagram illustrating a robot encountering a downward step, provided in an embodiment of this application. The robot fall environment determination method provided in this embodiment can be applied to... Figure 2 In the scenario shown. For example... Figure 2 As shown, 10 is the machine, 20 is the camera mounted on the robot 10, and the camera 20 is mounted on the upper part of the robot 10. The field of view of the camera 20 is part of the environment area in front of the robot 10. Z0 is the allowable error coordinate value. Z0 is less than 0. The coordinate value of each coordinate point in the set of coordinate points included in the fall point cloud data is less than Z0 on the Z-axis of the robot coordinate system. That is, the set of coordinate points included in the fall point cloud data is located below the horizontal plane where Z0 is located.

[0087] In this context, the camera is a device capable of directly or indirectly acquiring raw point cloud data. The raw point cloud data includes a set of coordinate points that represent the three-dimensional coordinates of multiple pixels within the field of view of the camera mounted on the robot, in the camera coordinate system. The camera coordinate system is a pre-set three-dimensional coordinate system, with the origin of the camera being the center of the camera. For example, the camera can be a depth camera or a stereo camera, etc., but this application embodiment does not limit this.

[0088] For example, a set of coordinate points includes multiple coordinate points. A robot can determine the set of coordinate points whose vertical coordinate values ​​are less than or equal to the allowable error coordinate values ​​from the multiple coordinate points included in the area point cloud data. This determined set of coordinate points constitutes the fall point cloud data. For instance, such as... Figure 2 As shown, the set of coordinate points in the regional point cloud data whose coordinate values ​​on the Z-axis of the robot coordinate system are less than the allowable error coordinate value Z0 is defined as the fall point cloud data.

[0089] As an example, before determining the fall point cloud data from the regional point cloud data, the robot can first acquire the raw point cloud data obtained by the camera, and then determine the regional point cloud data based on the raw point cloud data. For instance, the robot can convert the set of coordinate points in the raw point cloud data into a set of coordinate points in the robot coordinate system based on the transformation relationship between the camera coordinate system and the robot coordinate system. The converted set of coordinate points is the regional point cloud data.

[0090] As an example, the camera can acquire raw point cloud data of the camera's field of view in the environment in front of the robot at preset time intervals. Then, the robot can continuously acquire the raw point cloud data acquired by the camera when the acquisition conditions are met. Each time the robot acquires raw point cloud data, it executes the robot fall environment determination method provided in this embodiment. The acquisition conditions are preset conditions, such as a time interval twice the preset time interval. That is, for every two times the camera acquires raw point cloud data, the robot acquires raw point cloud data from the camera once, and the raw point cloud data acquired by the robot from the camera is the raw point cloud data acquired by the camera the second time.

[0091] Step 102: The robot divides the drop point cloud data into layers in the vertical direction to obtain multi-layered point cloud data.

[0092] In this multi-layered point cloud data, each layer of the point cloud data can include one or more coordinate points.

[0093] For example, the robot vertically layers the fall point cloud data into multiple layers, each with a preset height. The height of each layer in the vertical direction is the preset height. In other words, the robot layers the multiple coordinate points included in the fall point cloud data along the Z-axis of the robot's coordinate system, using the preset height as the unit. The preset height is a pre-set value, related to the step height. For instance, a step is typically 10cm-15cm, and in this embodiment, the preset height can be any value between 9cm-16cm, such as 9 or 13cm. Additionally, as... Figure 2 As shown, h is the preset height.

[0094] As an example, the preset height for stratifying the drop point cloud data corresponding to each acquired raw point cloud data can be the same or different. For instance, each time the robot acquires raw point cloud data, it determines the region point cloud data based on the acquired raw point cloud data, and then determines the drop point cloud data corresponding to the acquired raw point cloud data from the region point cloud data. For example, the preset height for stratifying the drop point cloud data corresponding to the first acquired raw point cloud data is 9, and the preset height for stratifying the drop point cloud data corresponding to the second acquired raw point cloud data is 13.

[0095] As an example, before layering the fall point cloud data in the vertical direction, the robot can first determine a second value, which indicates the number of coordinate points in the fall point cloud data whose vertical coordinate values ​​are less than the maximum height coordinate value. If the second value is less than a second preset number, then layerable point cloud data is determined from the fall point cloud data. Layerable point cloud data includes the set of coordinate points in the fall point cloud data whose vertical coordinate values ​​are greater than or equal to the maximum height coordinate value. Then, the robot layers the layerable point cloud data in the vertical direction to obtain multi-layered point cloud data. If the second value is greater than or equal to the second preset number, then the environment in front of the robot is determined to be a fall environment.

[0096] The maximum height coordinate value can be determined by a preset height and a tolerance coordinate value. The maximum height coordinate value refers to a coordinate value on the Z-axis of the robot coordinate system. The maximum height coordinate value is less than 0 and less than the tolerance coordinate value. For example, the maximum height coordinate value is equal to the sum of the preset height multiplied by a preset value, inverted, and the tolerance coordinate value. The preset value is a pre-set value. For example, ... Figure 2 As shown, Z1 is the maximum height coordinate value, h is the preset height, Z0 is the allowable error coordinate value, the preset value is 4, Z1=-(h×4)+Z0.

[0097] The second preset quantity is a pre-set value, for example, the second preset quantity is 100.

[0098] As an example, after the robot layers the fall point cloud data in the vertical direction, it can also obtain a single-layer layered point cloud data. It should be noted that, given the single-layer layered point cloud data, the specific implementation process for determining whether the environment in front of the robot is a fall environment will be described in detail in steps 103 and 104 below, and will not be repeated here in this embodiment.

[0099] Step 103: The robot determines the road conditions corresponding to each layer of the multi-layered point cloud data based on the point cloud data of each layer.

[0100] The road conditions corresponding to each layer of point cloud data include valid road conditions and invalid road conditions. Valid road conditions refer to road conditions that can be determined based on the layered point cloud data, while invalid road conditions refer to road conditions that cannot be determined based on the layered point cloud data.

[0101] For example, for the first layer of point cloud data in multi-layered point cloud data, if the number of coordinate points included in the first layer of point cloud data is greater than or equal to a first preset number, then the road condition corresponding to the first layer of point cloud data is determined to be a valid road condition, and the first layer of point cloud data is any layer of point cloud data in the multi-layered point cloud data; if the number of coordinate points included in the first layer of point cloud data is less than the first preset number, then the road condition corresponding to the first layer of point cloud data is determined to be an invalid road condition. Here, the first preset number is a pre-set value.

[0102] As an example, if the number of coordinate points included in the first layer of point cloud data is greater than or equal to a first preset number, the robot can first perform planar fitting on the first layer of point cloud data to obtain the fitting plane of the first layer of point cloud data, and then determine the type of valid road condition to which the road condition corresponding to the first layer of point cloud data belongs based on the first tilt angle of the fitting plane of the first layer of point cloud data. Here, the first tilt angle is the angle between the normal vector of the fitting plane of the first layer of point cloud data and the horizontal plane where the robot is located.

[0103] For example, the types of valid road conditions include at least steps, dangerous slopes, and ordinary slopes. If the first inclination angle is less than or equal to the first preset angle, the road condition corresponding to the first layer of point cloud data is determined to be a step; if the first inclination angle is greater than or equal to the second preset angle, the road condition corresponding to the first layer of point cloud data is determined to be a dangerous slope; if the first inclination angle is greater than the first preset angle and less than the second preset angle, the road condition corresponding to the first layer of point cloud data is determined to be an ordinary slope. Here, a dangerous slope refers to a slope that is difficult for the robot to traverse, such as one whose inclination angle is greater than or equal to the robot's maximum traversal angle. An ordinary slope refers to a slope that the robot can traverse. The second preset angle is greater than the first preset angle. The first preset angle is a pre-set angle that indicates the allowable error angle for the step, for example, the first preset angle can be 3 degrees. The second preset angle can be a pre-set angle, or it can be the robot's maximum traversal angle, for example, the second preset angle is the robot's maximum traversal angle of 16 degrees.

[0104] For example, the equation of the fitting plane for the first layer of point cloud data is expressed as Ax + By + Cz = D. Using the least squares method to fit the set of coordinate points included in the first layer of point cloud data to the plane, we determine A, B, C, and D in the equation, thus obtaining the fitting plane for the first layer of point cloud data. Its normal vector is expressed as... Then it can be based on the normal vector The first tilt angle is determined by the normal vector to the horizontal plane where the robot is located. For example, ... Figure 2 As shown, robot 10 divides the drop point cloud data into four layers, and then performs plane fitting on the first, second, and third layers of point cloud data to obtain the corresponding fitting planes.

[0105] As an example, before determining the type of valid road condition corresponding to the first layer of point cloud data based on the first tilt angle of the fitted plane of the first layer of point cloud data, the robot also determines whether the ratio of the number of coordinate points included in the fitted plane of the first layer of point cloud data to the total number of coordinate points included in the first layer of point cloud data is less than a ratio threshold. If it is less, the type of valid road condition corresponding to the first layer of point cloud data is determined to be an invalid slope; otherwise, the steps of determining the type of valid road condition corresponding to the first layer of point cloud data based on the first tilt angle of the fitted plane of the first layer of point cloud data and subsequent steps are executed. Here, the ratio threshold is a pre-set value, and an invalid slope refers to a situation where the ratio of the number of coordinate points included in the fitted plane determined based on the corresponding layer of point cloud data to the total number of coordinate points included in the corresponding layer of point cloud data is less than the ratio threshold; that is, the type of valid road condition also includes invalid slopes.

[0106] As an example, after obtaining the single-layer hierarchical point cloud data in step 102 above, the robot can perform planar fitting on the single-layer hierarchical point cloud data to obtain the corresponding fitting plane, and determine the road condition corresponding to the single-layer hierarchical point cloud data based on the tilt angle of the corresponding fitting plane. The specific implementation method of determining the road condition corresponding to the single-layer hierarchical point cloud data based on the tilt angle of the corresponding fitting plane is the same as the implementation method of determining the road condition corresponding to each layer of hierarchical point cloud data based on each layer of hierarchical point cloud data in a multi-layer hierarchical point cloud dataset, and will not be elaborated further in this embodiment.

[0107] Step 104: Based on the road conditions corresponding to the multi-layered point cloud data, the robot determines whether the environment ahead of the robot is a fall environment.

[0108] For example, the robot first determines a first value, which indicates the number of layers in the multi-layered point cloud data corresponding to a step-like road condition. If the first value is greater than or equal to a first quantity threshold, the robot's environment ahead is determined to be a fall environment. If the first value is equal to a second quantity threshold, the robot's environment ahead is determined to be a fall environment if the height of the first step corresponding to a step and the second tilt angle of the fitted plane of the multi-layered point cloud data corresponding to a step-like road condition satisfy a preset condition. If the first value is equal to 1, the robot's environment ahead is determined to be a fall environment if the number of layers in the multi-layered point cloud data corresponding to a dangerous slope is greater than or equal to a preset number of layers. Here, the first quantity threshold is greater than 1, the second quantity threshold is greater than 1 and less than the first quantity threshold, and the first quantity threshold, second quantity threshold, and preset number of layers are all pre-set values, for example, the first quantity threshold is equal to 3, the second quantity threshold is equal to 2, and the preset number of layers is equal to 2.

[0109] As an example, the robot can determine the height of a step by fitting a plane to the layered point cloud data corresponding to a step on the road.

[0110] As an example, the preset conditions are that the height of the first step corresponding to the road condition of the step in the multi-layered point cloud data is arranged in an arithmetic sequence, and the difference of the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to the road condition of the step is less than the allowable error angle, which is a preset error angle.

[0111] Additionally, if the first value equals 1, then in the case where all road conditions other than steps in the multi-layered point cloud data are invalid road conditions, the robot determines the second step height corresponding to the step in the multi-layered point cloud data. If the second step height is greater than or equal to the height threshold, then the robot's forward environment is determined to be a fall environment. If the second step height is less than the height threshold, then the robot is controlled to rotate by a third preset angle. If, based on the regional point cloud data within the field of view of the robot's camera after rotation, it is determined that there is no ordinary slope in the forward environment of the robot after rotation, then the forward environment of the robot before rotation is determined to be a fall environment.

[0112] The height threshold can be pre-determined based on the robot's maximum travel angle, the nearest blind zone distance, and the camera's installation height. The installation height is the vertical distance between the origin of the camera's coordinate system and the origin of the robot's coordinate system. The nearest blind zone distance is the distance between the nearest blind zone point and the origin of the robot's coordinate system. The nearest blind zone point is the X-axis coordinate of the point where the camera's nearest blind zone intersects with the robot's X-axis. The third preset angle is a pre-set angle. The camera has a certain field of view; the camera's nearest blind zone refers to the area between the camera's field of view and the robot body along the robot's travel direction. For example, ... Figure 2 The nearest blind spot area shown, such as Figure 2 As shown, G is the location of the nearest blind spot.

[0113] As an example, after determining the road condition corresponding to the single-layer hierarchical point cloud data based on the tilt angle of the corresponding fitting plane in step 103, the robot can also determine the second step height of the single-layer hierarchical point cloud data if the road condition corresponding to the single-layer hierarchical point cloud data is a step, and if the distance between the edge of the step and the center of the robot is equal to the nearest blind zone distance of the camera. Then, it can determine whether the second step height is less than the height threshold.

[0114] In this embodiment, fall point cloud data is first determined from regional point cloud data. Then, the fall point cloud data is layered vertically to obtain multi-layered point cloud data. Next, based on each layer of the multi-layered point cloud data, the road conditions corresponding to each layer are determined. Finally, based on the road conditions corresponding to the multi-layered point cloud data, it is determined whether the environment ahead of the robot is a fall environment. The regional point cloud data includes a set of coordinate points that represent the set of three-dimensional coordinates of multiple pixels within the field of view of the camera mounted on the robot, in the robot's coordinate system. In this way, the road conditions corresponding to the multi-layered point cloud data obtained after layering the fall point cloud can be determined. Based on the different combinations of road conditions corresponding to different layers of point cloud data, it can be determined whether the environment in front of the robot is a fall environment. That is, based on each layer of point cloud data obtained after layering the fall point cloud, the road conditions corresponding to each layer of point cloud data can be determined. Based on the different combinations of road conditions corresponding to different layers of fall point cloud data, it can be determined whether the environment in front of the robot is a fall environment. This reduces the dependence on the correlation between all coordinate points included in the fall point cloud data when determining the fall environment, improves the accuracy of determining the fall environment, and reduces the probability of the robot falling or stopping erroneously.

[0115] Please refer to Figure 3 , Figure 3This is a flowchart of another method for determining a robot fall environment provided in an embodiment of this application. This method can be applied to a robot equipped with a camera, or to a computer device connected to the robot. The computer device can be a terminal, server, or embedded device, and the terminal can be a desktop computer or tablet computer. The following explanation will use a robot equipped with a camera as an example to illustrate the robot fall environment determination method. Figure 3 As shown, the method includes the following steps:

[0116] Step 301: The robot determines the fall point cloud data from the regional point cloud data.

[0117] The regional point cloud data includes a set of coordinate points that are the set of three-dimensional coordinate points corresponding to multiple pixels in the field of view of the camera mounted on the robot in the robot coordinate system. The fall point cloud data includes a set of coordinate points in the regional point cloud data whose vertical coordinate values ​​are less than or equal to the allowable error coordinate values, which are preset values.

[0118] Step 302: The robot divides the drop point cloud data into layers in the vertical direction to obtain multi-layered point cloud data.

[0119] In this multi-layered point cloud data, each layer of the point cloud data can include one or more coordinate points.

[0120] As an example, such as Figure 2As shown, Z0 is the allowable error coordinate value, and Z1 is the maximum height coordinate value. Before the robot layers the fall point cloud data in the vertical direction, a second value can be determined. This second value indicates the number of coordinate points in the fall point cloud data whose vertical coordinate value is less than the maximum height coordinate value. In other words, it indicates the number of coordinate points in the fall point cloud data whose coordinate value on the Z-axis of the robot coordinate system is less than Z1. If the second value is less than a second preset number, it means that the fall point cloud data includes fewer coordinate points whose coordinate value on the Z-axis of the robot coordinate system is less than Z1, i.e., fewer coordinate points located below the horizontal plane where Z1 is located. Since the acquired fall point cloud data may contain errors, in this case, it is insufficient to determine whether the environment in front of the robot is a fall environment based on the fewer coordinate points in the fall point cloud data whose coordinate value on the Z-axis of the robot coordinate system is less than Z1. Based on this, if the second value is less than the second preset quantity, the robot can first determine the layerable point cloud data from the fall point cloud data. The layerable point cloud data includes the set of coordinate points in the fall point cloud data whose vertical coordinate values ​​are greater than or equal to the maximum height coordinate value. That is, the coordinate points included in the layerable point cloud data are located below the horizontal plane Z0 and above the horizontal plane Z1. Then, the robot layers the layerable point cloud data in the Z-axis direction of the robot coordinate system, using a preset height h as the unit, to obtain 4 layers of layered point cloud data. Among them, the maximum height coordinate value can be determined by the preset height and the allowable error coordinate value, and the second preset quantity is a pre-set value.

[0121] Additionally, if the second value is greater than or equal to the second preset quantity, it indicates that the fall point cloud data includes a large number of coordinate points whose coordinate values ​​on the Z-axis of the robot coordinate system are less than Z1. In other words, the fall point cloud data includes a large number of coordinate points located below the horizontal plane where Z1 is located. In this case, it can be determined that the road condition corresponding to the fall point cloud data is a series of steps or deep potholes, thus determining that the environment in front of the robot is a fall environment.

[0122] Step 303: The robot determines the number of layers K of the multi-layered point cloud data and sets the number of iterations m to be equal to 1.

[0123] Wherein, the number of layers K is a positive integer greater than 1, and the number of cycles m is a positive integer.

[0124] Step 304: The robot determines whether the number of loops m is greater than the number of layers K. If the number of loops m is greater than K, then proceed to step 309; otherwise, proceed to step 305.

[0125] For example, if the number of iterations m is less than or equal to the number of layers K, then at least one layer of the multi-layered point cloud data has undetermined road conditions. If the number of iterations m is greater than the number of layers K, then the road conditions corresponding to each layer of the multi-layered point cloud data have been determined, that is, the road conditions corresponding to each layer of the multi-layered point cloud data have been obtained.

[0126] Step 305: The robot determines whether the number of coordinate points included in the m-th layer point cloud data is less than the first preset number. If it is less, proceed to step 306; otherwise, proceed to step 307.

[0127] The first preset quantity is a pre-set value.

[0128] Among them, the road conditions corresponding to each layer of the multi-layered point cloud data include valid road conditions and invalid road conditions. Valid road conditions refer to the road conditions corresponding to the layered point cloud data that can be determined based on the layered point cloud data, while invalid road conditions refer to the road conditions corresponding to the layered point cloud data that cannot be determined based on the layered point cloud data.

[0129] For example, if the number of coordinate points included in the m-th layer point cloud data is less than the first preset number, it means that the number of coordinate points included in the m-th layer point cloud data is too small, and it is impossible to perform planar fitting on the m-th layer point cloud data based on the coordinate points included in the m-th layer point cloud data. Therefore, the road condition corresponding to the m-th layer point cloud data is determined as an invalid road condition. If the number of coordinate points included in the m-th layer point cloud data is greater than or equal to the first preset number, it means that the number of coordinate points included in the m-th layer point cloud data is too large, and it is possible to perform planar fitting on the m-th layer point cloud data. Thus, the road condition corresponding to the m-th layer point cloud data is determined as a valid road condition.

[0130] Step 306: Determine that the road condition corresponding to the m-th layer point cloud data is an invalid road condition, update the loop count m to m+1, and jump to step 304.

[0131] Step 307: The robot performs plane fitting on the m-th layer point cloud data to obtain the fitting plane of the m-th layer point cloud data.

[0132] For example, the equation of the fitting plane for the m-th layer point cloud data is Ax + By + Cz = D. The robot uses the least squares method to fit the set of coordinate points included in the m-th layer point cloud data to the plane, determines A, B, C and D in the equation, and obtains the fitting plane for the m-th layer point cloud data.

[0133] Step 308: The robot determines the type of valid road condition corresponding to the m-th layer point cloud data based on the first tilt angle of the fitting plane of the m-th layer point cloud data, updates the loop count m to m+1, and jumps to step 304. The types of valid road conditions include at least steps, dangerous slopes and ordinary slopes.

[0134] Wherein, the first tilt angle is the angle between the normal vector of the fitting plane of the first layered point cloud data and the horizontal plane where the robot is located.

[0135] For example, the robot can determine its first tilt angle based on the angle between the normal vector of the fitting plane Ax+By+Cz=D of the m-th layer of point cloud data and the normal vector of the robot's horizontal plane. For instance, the normal vector of the fitting plane Ax+By+Cz=D can be expressed as... The robot first determines the normal vector. The angle between the robot and the normal vector of the horizontal plane, the normal vector The complementary angle between the angle between the robot and the normal vector of the horizontal plane on which the robot is located is the first tilt angle.

[0136] It should be noted that after repeating steps 304-308 K times, the road conditions corresponding to each layer of the multi-layered point cloud data can be obtained.

[0137] Step 309: The robot determines a first value, which is used to indicate the number of layers in the multi-layered point cloud data corresponding to the road condition of steps.

[0138] The first value is an integer, which is less than or equal to the number of layers K and greater than or equal to 0.

[0139] Step 310: The robot determines whether the first value is less than the first quantity threshold. If the first value is less than the first quantity threshold, then step 311 is executed; otherwise, step 320 is executed.

[0140] The first quantity threshold is greater than 1, and the first quantity threshold is a preset value.

[0141] For example, if the first quantity threshold is 3, and the first value is greater than or equal to 3, it means that after the drop point cloud data is layered, at least 3 layers of the multi-layered point cloud data correspond to the road condition of steps. In this case, the probability of incorrectly determining that the road condition corresponding to the at least 3 layers of the multi-layered point cloud data is steps is low. Therefore, it can be directly determined that the road condition corresponding to the drop point cloud data is steps, and the environment in front of the robot is a drop environment.

[0142] Step 311: The robot determines whether the first value is equal to the second quantity threshold. If the first value is equal to the second quantity threshold, then proceed to step 312; otherwise, proceed to step 313.

[0143] The second quantity threshold is greater than 1 and less than the first quantity threshold, and the second quantity threshold is a preset value.

[0144] For example, if the second quantity threshold is 2, and the first value is equal to 2, it means that after the drop point cloud data is layered, there are 2 layers of layered point cloud data in the multi-layered point cloud data corresponding to the road condition of steps. Otherwise, it means that after the drop point cloud data is layered, there are 1 or 0 layers of layered point cloud data in the multi-layered point cloud data corresponding to the road condition of steps.

[0145] Step 312: The robot determines whether the height of the first step corresponding to the road condition of the step in the multi-layered point cloud data and the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to the road condition of the step meet the preset conditions. If the preset conditions are met, step 320 is executed; otherwise, step 321 is executed.

[0146] The preset conditions are that the height of the first step corresponding to the road condition step in the multi-layered point cloud data is arranged in an arithmetic sequence, and the difference of the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to the road condition step is less than the allowable error angle, which is a preset angle.

[0147] For example, the second quantity threshold is 2. In the multi-layered point cloud data, there are 2 layers of point cloud data corresponding to the road condition of steps. The preset condition is that the heights of the two first steps corresponding to the two layers of point cloud data corresponding to the road condition of steps are arranged in an arithmetic sequence, and the difference between the second tilt angles of the two fitted planes corresponding to the two layers of point cloud data corresponding to the road condition of steps is less than the allowable error angle.

[0148] For example, in a multi-layered point cloud data, if the road condition corresponding to two layers of point cloud data is a step, and if the height of the first step corresponding to the step in the two layers of point cloud data and the second tilt angle of the fitting plane of the point cloud data corresponding to the step in the two layers of point cloud data meet the preset conditions, then the probability of both layers of point cloud data being incorrectly identified as steps is low. It can be directly determined that the road condition corresponding to the fall point cloud data is a step, and the environment in front of the robot is a fall environment.

[0149] Step 313: The robot determines whether the first value is equal to 1. If the first value is equal to 1, then proceed to step 314; otherwise, proceed to step 321.

[0150] If the first value is equal to 1, then after layering the fall point cloud data, there is a layer of point cloud data corresponding to a step as the road condition. If the first value is not equal to 1, then after layering the fall point cloud data, there is no layer of point cloud data corresponding to a step as the road condition, and it is determined that the environment in front of the robot is not a fall environment.

[0151] Step 314: The robot determines whether the number of layers in the multi-layer point cloud data corresponding to the dangerous slope is less than the preset number of layers. If it is less, proceed to step 315; otherwise, proceed to step 320.

[0152] The preset number of layers is a pre-set value, for example, the preset number of layers is 2.

[0153] Step 315: The robot determines whether all road conditions other than steps in the multi-layer point cloud data are invalid. If so, proceed to step 316; otherwise, proceed to step 321.

[0154] For example, please refer to Figure 4 , Figure 4 This is a schematic diagram of a multi-layered point cloud data where the number of layers corresponding to the road condition of steps is 1, provided in an embodiment of this application.

[0155] like Figure 4 As shown, if the road condition corresponding to one layer of multi-layered point cloud data is a step, and the road conditions corresponding to other layers of point cloud data are invalid, then the road condition corresponding to the first layer of point cloud data may be incorrectly identified as a step. For example, if there is a common slope in the nearest blind zone of the camera, and this common slope in the nearest blind zone, together with the step in the road condition corresponding to the first layer of point cloud data (the step within the camera's field of view), constitutes a common slope, meaning the step within the camera's field of view is incorrectly identified. To avoid this situation, when all road conditions in the multi-layered point cloud data are invalid except for the step, the robot performs steps 316-319 to further determine whether a common slope exists in the nearest blind zone of the camera, thereby improving the accuracy of determining the fall environment.

[0156] As an example, after determining that all road conditions other than steps in the multi-layered point cloud data are invalid road conditions, the robot also determines whether the distance between the edge of the step corresponding to the step in the multi-layered point cloud data and the center of the robot is equal to the nearest blind zone distance of the camera. If so, step 316 is executed; otherwise, step 321 is executed.

[0157] The nearest blind zone distance refers to the distance between the nearest blind zone point and the origin of the robot coordinate system. The nearest blind zone point is the coordinate value on the X-axis of the point where the nearest blind zone area of ​​the camera intersects with the X-axis of the robot coordinate system.

[0158] For example, such as Figure 4As shown, A is the origin of the camera coordinate system, B is the origin of the robot coordinate system, G is the location of the nearest blind spot point, BG is the nearest blind spot distance, and H is the edge of the step corresponding to the road condition of a step in the multi-layered point cloud data. The distance between the edge of the step corresponding to the road condition of a step in the multi-layered point cloud data and the center of the robot is equal to the nearest blind spot distance of the camera, which means that G and H coincide and BH equals BG. In this case, the distance between the robot's location and the edge of the step corresponding to the road condition of a step in the multi-layered point cloud data is equal to the nearest blind spot distance.

[0159] For example, if the robot determines that all road conditions in the multi-layered point cloud data, except for steps, are not invalid, it executes step 321. If it determines that all road conditions in the multi-layered point cloud data, except for steps, are invalid, it determines whether the distance between the edge of the step corresponding to the step in the multi-layered point cloud data and the robot's center is equal to the nearest blind zone distance of the camera. If so, it executes step 316; otherwise, it executes step 321. Thus, when all road conditions in the multi-layered point cloud data, except for steps, are invalid, and the robot's location is greater than the nearest blind zone distance from the edge of the step corresponding to the step in the multi-layered point cloud data, the robot's forward movement over a short distance (during travel distance less than the blind zone distance) is safe. Therefore, provided the robot does not fall, it can continue moving forward.

[0160] It should be noted that, in order to facilitate the distinction between single-layer and multi-layer steps in a fall environment, steps 315-319 in the embodiments of this application can be referred to as the robot fall environment determination method for single-layer steps, and correspondingly steps 310-314 can be referred to as the robot fall environment determination method for multi-layer steps.

[0161] Step 316: The robot determines the height of the second step corresponding to the road condition in the multi-layered point cloud data.

[0162] In this multi-layered point cloud data, one layer corresponds to a step in the road condition. The robot can determine the height of the second step based on the fitted plane of this first-layered point cloud data. For example, the equation of the fitted plane of this first-layered point cloud data is Ax + By + Cz = D, where D is the height of the second step.

[0163] Step 317: The robot determines whether the height of the second step is less than the height threshold. If it is less, proceed to step 318; otherwise, proceed to step 320.

[0164] The height threshold can be determined based on the robot's maximum travel angle, the nearest blind spot distance, and the camera's installation height.

[0165] As an example, before determining whether the height of the second step is less than the height threshold, the robot first determines the height threshold based on the robot's maximum travel angle, the nearest blind spot distance, and the camera's installation height. For instance, the robot determines the height threshold based on the maximum travel angle, the nearest blind spot distance, and the installation height using the following formula (1):

[0166]

[0167] Where r is the height threshold, AB is the camera's installation height, and BH is the nearest blind zone distance. This represents the robot's maximum travel angle.

[0168] As an example, if the camera's installation height AB is 85, the nearest blind spot distance BH is 62, and the maximum pass-through angle... If the height is 16 degrees, then the height threshold r is 22.8.

[0169] For example, such as Figure 4 As shown, the multi-layered point cloud data has only one layer of point cloud data corresponding to the road condition of steps. A is the origin of the camera coordinate system, B is the origin of the robot coordinate system, H is the location of the nearest blind spot point, BH is the nearest blind spot distance, ABH is the nearest blind spot area of ​​the camera, and AB is the installation height of the camera. Assuming that there is a slope EF in the nearest blind spot area of ​​the camera's field of view, and the tilt angle of the slope EF is θ, the slope height DF of the slope EF can be expressed by the following formula (2):

[0170]

[0171] Since the distance B from the edge of the step corresponding to the road condition of the step in the multi-layer point cloud data to the center of the robot is equal to the nearest blind zone distance, H is the edge of the step corresponding to the road condition of the step in the 1st layer point cloud data, EL is the fitting plane of the 1st layer point cloud data corresponding to the road condition of the step, HE is the step within the field of view of the camera, HI is the step height (second step height) of the step corresponding to the road condition of the step in the 1st layer point cloud data, B coincides with G, C coincides with D, so DI = BH. The above formula (2) can be expressed as the following formula (3):

[0172]

[0173] If the inclination angle θ of the inclined plane EF is equal to the maximum passage angle Maximum passing angle The corresponding slope height DF is equal to the second step height HI, i.e., DF = HI, then the maximum passage angle is... The corresponding slope height DF is the maximum slope height that may exist within the nearest blind zone of the camera's field of view. Thus, the maximum slope height that may exist within the nearest blind zone of the camera's field of view can be expressed by the following formula (4):

[0174]

[0175] Where DF′ is the maximum height of a common slope that may exist within the nearest blind zone of the camera's field of view, i.e., the height threshold r, AB is the camera's installation height, and BH is the nearest blind zone distance. This represents the robot's maximum travel angle.

[0176] For example, if the height HI of the second step is greater than or equal to the maximum slope height DF′, then it is determined that there is no tilt angle less than the maximum passage angle within the camera's nearest blind zone. The ordinary inclined plane, together with the steps within the field of view, forms an ordinary inclined plane. Therefore, the accuracy of this layer 1 point cloud data in identifying the road condition as steps is relatively high, confirming that the environment in front of the robot is a fall environment. Furthermore, to further ensure the robot's safety, the height threshold r can be set to a value greater than the maximum inclined plane height DF′.

[0177] For example, if the height HI of the second step is less than the maximum slope height DF′, then there may be areas in the camera's nearest blind zone where the tilt angle is less than the maximum passage angle. The robot performs steps 318-319 to further determine whether a common slope exists in the camera's nearest blind zone.

[0178] Step 318: The robot rotates to the third preset angle.

[0179] The third preset angle is a pre-set angle, such as 10 degrees or 20 degrees.

[0180] Step 319: The robot determines whether there is a common slope in the environment in front of the rotated robot. If there is no common slope, proceed to step 320; otherwise, proceed to step 321.

[0181] For example, in a multi-layered point cloud dataset, if the first layer corresponds to a step, the robot uses the point cloud data within the field of view of its rotated camera to determine if a common slope exists in the environment ahead. If no common slope exists, then the robot's camera's nearest blind spot area before rotation also doesn't contain a common slope. The accuracy of identifying the step as the first layer of point cloud data is high, thus determining the robot's environment ahead to be a fall environment. Conversely, if a common slope exists, then the robot's camera's nearest blind spot area before rotation also contains a common slope. The accuracy of identifying the step as the first layer of point cloud data is low, thus determining the robot's environment ahead to be a fall environment.

[0182] As an example, the robot can first determine the fall point cloud data within the field of view of the rotated camera based on the regional point cloud data within the field of view of the rotated robot. Then, it can perform planar fitting on the fall point cloud data within the field of view of the rotated camera to obtain the fitting plane of the fall point cloud data within the field of view of the rotated camera. If the third tilt angle of the fitting plane of the fall point cloud data within the field of view of the rotated camera is less than or equal to the first preset angle, or the third tilt angle is greater than or equal to the second preset angle, it is determined that there is no ordinary slope in the environment in front of the rotated robot. Otherwise, it is determined that there is an ordinary slope in the environment in front of the rotated robot.

[0183] Step 320: The robot determines that the environment in front of it is a fall environment.

[0184] As an example, once a robot determines that the environment ahead is a fall-prone environment, it can stop itself or report a fault code to the computer device connected to it. This allows the computer device to control the robot's brakes or other mechanisms to prevent the robot from falling and ensure its safety.

[0185] As an example, after determining that the environment ahead of the robot is a fall environment, the fall point cloud data can be zoomed in the opposite direction of the robot's movement by a first preset zoom value. This allows the fitted plane of the fall point cloud data after zooming in to act as a virtual wall, ensuring the robot cannot continue forward when approaching the virtual wall. This avoids situations where the robot has to traverse steps or deep pits in the fall environment due to the robot failing to stop itself or the computer system failing to brake, further ensuring the robot's safety. The first preset zoom value is a pre-set value that is less than the nearest blind zone distance. Zooming in the fall point cloud data in the opposite direction of the robot's movement by the first preset zoom value means subtracting the first preset zoom value from the coordinates of the multiple coordinate points included in the fall point cloud data on the X-axis of the robot's coordinate system.

[0186] Step 321: The robot determines that the environment in front of it is not a fall environment.

[0187] As an example, after determining that the environment ahead of the robot is not a fall-prone environment, it can further implement different processing schemes for the fall point cloud data or the layered point cloud data based on the road conditions corresponding to the multi-layered point cloud data, in order to prevent the robot from falling and further ensure its safety. For example, the processing schemes include filtering, retaining, and retaining the fall point cloud data or the layered point cloud data while pulling it closer to a second preset value in the opposite direction of the robot's movement. The second preset value is a pre-set value, and the first and second preset values ​​can be the same or different.

[0188] For example, in step 312, if it is determined that the height of the first step corresponding to a step in the multi-layered point cloud data and the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to a step do not meet the preset conditions, then after determining that the environment in front of the robot is not a fall environment, the multi-layered point cloud data corresponding to a dangerous slope is pulled closer to the robot's direction of travel by a second preset pulling value. This uses the fitting plane of the corresponding multi-layered point cloud data after being pulled closer to the second preset pulling value as a virtual wall, preventing the robot from passing through the dangerous slope and preventing a fall, thus further ensuring the robot's safety. Alternatively, the multi-layered point cloud data corresponding to ordinary slopes can be filtered, i.e., deleted, to reduce the amount of data stored in the robot.

[0189] Alternatively, in step 315, if it is determined that the distance between the edge of the step corresponding to the road condition of the step in the multi-layer point cloud data and the center of the robot is not equal to the nearest blind zone distance of the camera, then after determining that the environment in front of the robot is not a fall environment, the multi-layer point cloud data corresponding to the road condition of the step is pulled closer to the robot's direction of travel by a second preset zoom value, so as to ensure that the robot does not fall by fitting the plane of the multi-layer point cloud data after the second preset zoom value is pulled closer, and to ensure the safety of the robot continuing to move forward.

[0190] Alternatively, in step 313, if the first value is determined to be equal to 1, then the layer number of the layered point cloud data corresponding to the road condition of the step is equal to 0. After determining that the environment in front of the robot is not a fall environment, the fall point cloud data determined in this instance is retained. This allows the robot to perform steps 302-321 based on the fall point cloud data determined in this instance and the fall point cloud data determined in the next instance after determining the fall point cloud data from the regional point cloud data. This determines whether the environment in front of the robot is a fall environment, further improving the accuracy of determining the fall environment and reducing the probability of the robot falling or stopping erroneously.

[0191] It should be noted that in this embodiment, the camera can acquire raw point cloud data of the camera's field of view of the environment in front of the robot at preset time intervals. Then, the robot can continuously acquire the raw point cloud data acquired by the camera when the acquisition conditions are met, determine the regional point cloud data based on the raw point cloud data, and then execute steps 301-321 in this embodiment. That is, the robot can continuously determine the regional point cloud data when the acquisition conditions are met, and execute steps 301-321 in this embodiment once each time the robot determines the regional point cloud data. In this case, it is possible that for the first determined regional point cloud data, after executing steps 301-321, it is determined that the environment in front of the robot is not a fall environment; for the second determined regional point cloud data, after executing steps 301-321, it is determined that the environment in front of the robot is a fall environment. Thus, by continuously determining the regional point cloud data and executing steps 301-321, the accuracy of determining whether the environment in front of the robot is a fall environment is improved, reducing the probability of the robot falling or stopping unnecessarily.

[0192] In this embodiment, the road conditions corresponding to each layer of the multi-layered point cloud data obtained after layering the fall point cloud can be determined. Based on different combinations of road conditions corresponding to different layers of fall point cloud data, it can be determined whether the environment in front of the robot is a fall environment. This reduces the dependence on the correlation between all coordinate points included in the fall point cloud data when determining the fall environment, improves the accuracy of determining the fall environment, and reduces the probability of the robot falling or stopping erroneously.

[0193] Figure 5 This is a schematic diagram of a robot fall environment determination device provided in an embodiment of this application. This robot fall environment determination device can be implemented by software, hardware, or a combination of both, forming part or all of a computer device, which can be described below. Figure 6 The computer equipment shown. See also Figure 5 The device includes: a first determining module 501, a layering module 502, a second determining module 503, and a third determining module 504.

[0194] The first determining module 501 is used to determine the fall point cloud data from the regional point cloud data. The regional point cloud data includes a set of coordinate points that is a set of three-dimensional coordinates of multiple pixels in the field of view of the camera mounted on the robot in the robot coordinate system.

[0195] The layering module 502 is used to layer the drop point cloud data in the vertical direction to obtain multi-layered point cloud data;

[0196] The second determining module 503 is used to determine the road conditions corresponding to each layer of the multi-layer point cloud data based on each layer of the multi-layer point cloud data.

[0197] The third determination module 504 is used to determine whether the environment in front of the robot is a fall environment based on the road conditions corresponding to the multi-layered point cloud data.

[0198] As an example, the second determining module 503 is also used to determine the road condition corresponding to the first layer point cloud data as a valid road condition if the number of coordinate points included in the first layer point cloud data in the multi-layer point cloud data is greater than or equal to the first preset number, and the first layer point cloud data is any layer point cloud data of the multi-layer point cloud data.

[0199] If the number of coordinate points included in the first layer of point cloud data is less than the first preset number, then the road condition corresponding to the first layer of point cloud data is determined to be an invalid road condition.

[0200] As an example, the second determining module 503 is also used to perform plane fitting on the first layered point cloud data if the number of coordinate points included in the first layered point cloud data is greater than or equal to the first preset number, so as to obtain the fitting plane of the first layered point cloud data.

[0201] Based on the first tilt angle of the fitting plane of the first layer point cloud data, the type of valid road condition corresponding to the road condition of the first layer point cloud data is determined. The first tilt angle is the angle between the normal vector of the fitting plane of the first layer point cloud data and the horizontal plane where the robot is located.

[0202] As an example, valid road condition types include at least steps, dangerous slopes, and ordinary slopes;

[0203] The second determining module 503 is further configured to determine the road condition corresponding to the first layered point cloud data as a step if the first tilt angle is less than or equal to the first preset angle.

[0204] If the first tilt angle is greater than or equal to the second preset angle, then the road condition corresponding to the first layered point cloud data is determined to be the dangerous slope, and the second preset angle is greater than the first preset angle.

[0205] If the first tilt angle is greater than the first preset angle and less than the second preset angle, then the road condition corresponding to the first layer point cloud data is determined to be a normal slope.

[0206] As an example, the road conditions corresponding to multi-layered point cloud data include at least steps and dangerous slopes;

[0207] The third determining module 504 is also used to determine a first value, which is used to indicate the number of layers in the multi-layered point cloud data corresponding to the road condition of steps.

[0208] If the first value is greater than or equal to the first quantity threshold, then the environment in front of the robot is determined to be a fall environment, and the first quantity threshold is greater than 1.

[0209] If the first value is equal to the second quantity threshold, then if the height of the first step corresponding to the road condition of the step in the multi-layered point cloud data and the second tilt angle of the fitting plane of the multi-layered point cloud data corresponding to the road condition of the step satisfy the preset conditions, the environment in front of the robot is determined to be a fall environment, and the second quantity threshold is greater than 1 and less than the first quantity threshold.

[0210] If the first value is equal to 1, then if the number of layers of the layered point cloud data corresponding to the dangerous slope in the multi-layered point cloud data is greater than or equal to the preset number of layers, the environment in front of the robot is determined to be a fall environment.

[0211] As an example, the road conditions corresponding to multi-layered point cloud data also include invalid road conditions and ordinary slopes;

[0212] The third determining module 504 is also used to determine the second step height of the road condition corresponding to the step in the multi-layered point cloud data if the first value is equal to 1, and other road conditions other than the step are invalid road conditions in the road conditions corresponding to the multi-layered point cloud data.

[0213] If the height of the second step is greater than or equal to the height threshold, then the environment in front of the robot is determined to be a fall environment.

[0214] If the height of the second step is less than the height threshold, the robot is controlled to rotate by a third preset angle. If, based on the point cloud data of the area within the field of view of the robot's camera after rotation, it is determined that there is no ordinary slope in the environment in front of the robot after rotation, then the environment in front of the robot before rotation is determined to be a fall environment.

[0215] As an example, the layering module 502 is also used to layer the drop point cloud data in the vertical direction in units of a preset height to obtain multi-layered point cloud data, where the height of each layer of point cloud data in the vertical direction is a preset height.

[0216] As an example, the device further includes a fourth determining module, a fifth determining module, and a sixth determining module;

[0217] The fourth determining module is used to determine the second value, which indicates the number of coordinate points in the drop point cloud data whose vertical coordinate values ​​are less than the maximum height coordinate value.

[0218] The fifth determining module is used to determine layerable point cloud data from the drop point cloud data if the second value is less than the second preset number. The layerable point cloud data includes a set of coordinate points in the drop point cloud data whose vertical coordinate values ​​are greater than or equal to the maximum height coordinate value.

[0219] The layering module 502 is also used to layer the layerable point cloud data in the vertical direction to obtain multi-layered point cloud data.

[0220] The sixth determining module is used to determine that the environment in front of the robot is a fall environment if the second value is greater than or equal to the second preset quantity.

[0221] It should be noted that the robot fall environment determination device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0222] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0223] The robot fall environment determination device and the robot fall environment determination method provided in the above embodiments belong to the same concept. The specific working process and technical effects of the units and modules in the above embodiments can be found in the method embodiment section, and will not be repeated here.

[0224] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 6 As shown, the computer device includes: a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the robot fall environment determination method in the above embodiments.

[0225] The computer equipment can be the above. Figure 1 Examples or the above Figure 3 The computer device in the embodiments, or the one described above. Figure 1 Examples or the above Figure 3 The robot in this embodiment. The computer device may be a near-eye display device, or a desktop computer, laptop computer, web server, handheld computer, mobile phone, tablet computer, wireless terminal device, communication device, or embedded device. This application embodiment does not limit the type of computer device. Those skilled in the art will understand that... Figure 6 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as input / output devices, network access devices, etc.

[0226] Processor 601 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0227] In some embodiments, memory 602 may be on-chip memory or off-chip memory of a computer device, such as cache memory, SRAM (Static Random-Access Memory), DRAM (Dynamic Static Random-Access Memory), or floppy disk. In other embodiments, memory 602 may be a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card provided on the computer device. Furthermore, memory 602 may include internal storage units of on-chip and off-chip memory of the computer device, as well as external storage devices. Memory 602 is used to store the operating system, applications, boot loader, data, and other programs. Memory 602 can also be used to temporarily store data that has been output or will be output.

[0228] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0229] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0230] This application provides a computer program product that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.

[0231] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above method embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, and optical data storage devices. The computer-readable storage medium mentioned in this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0232] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented in whole or in part as a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0233] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A robot fall environment determination method characterized by, The method comprises: determining drop point cloud data from area point cloud data, the area point cloud data comprising a set of coordinate points of a plurality of pixel points in a field of view of a camera carried by a robot in a robot coordinate system; layering the drop point cloud data in a vertical direction to obtain multi-layered point cloud data; determining a road condition corresponding to each layer of the multi-layered point cloud data according to each layer of the multi-layered point cloud data; determining whether an environment in front of the robot is a drop environment according to the road conditions corresponding to the multi-layered point cloud data; the road conditions corresponding to the multi-layered point cloud data at least include a step and a dangerous slope; the determining whether the environment in front of the robot is the drop environment according to the road conditions corresponding to the multi-layered point cloud data comprises: determining a first value, the first value being used to indicate a number of layers of the multi-layered point cloud data corresponding to the step; if the first value is greater than or equal to a first quantity threshold, determining that the environment in front of the robot is the drop environment, the first quantity threshold being greater than 1; if the first value is equal to a second quantity threshold, determining that the environment in front of the robot is the drop environment in a case where a first step height corresponding to the step in the multi-layered point cloud data and a second inclination angle of a fitting plane of the multi-layered point cloud data corresponding to the step satisfy a preset condition, the second quantity threshold being greater than 1 and less than the first quantity threshold; if the first value is equal to 1, determining that the environment in front of the robot is the drop environment in a case where a number of layers of the multi-layered point cloud data corresponding to the dangerous slope is greater than or equal to a preset layer number; the road conditions corresponding to the multi-layered point cloud data further include an invalid road condition and an ordinary slope, and the method further comprises: if the first value is equal to 1, determining a second step height corresponding to the step in the multi-layered point cloud data in a case where other road conditions in the road conditions corresponding to the multi-layered point cloud data except for the step are the invalid road conditions; if the second step height is greater than or equal to a height threshold, determining that the environment in front of the robot is the drop environment; if the second step height is less than the height threshold, controlling the robot to rotate by a third preset angle, and if it is determined according to area point cloud data in a field of view of the camera of the robot after rotation that the environment in front of the robot after rotation is free of the ordinary slope, determining that the environment in front of the robot before rotation is the drop environment.

2. The method of claim 1, wherein, the determining the road condition corresponding to each layer of the multi-layered point cloud data according to each layer of the multi-layered point cloud data comprises: For the first layered point cloud data in the multi-layered point cloud data, if the first layered point cloud data includes a number of coordinate points greater than or equal to a first preset number, it is determined that the road condition corresponding to the first layered point cloud data is a valid road condition, and the first layered point cloud data is any layer of the multi-layered point cloud data; If the first layered point cloud data includes a number of coordinate points less than the first preset number, it is determined that the road condition corresponding to the first layered point cloud data is an invalid road condition.

3. The method of claim 2, wherein, If the first layered point cloud data includes a number of coordinate points greater than or equal to the first preset number, it is determined that the road condition corresponding to the first layered point cloud data is a valid road condition, including: If the first layered point cloud data includes a number of coordinate points greater than or equal to the first preset number, a plane fitting is performed on the first layered point cloud data to obtain a fitting plane of the first layered point cloud data; According to a first inclination angle of the fitting plane of the first layered point cloud data, a type of the valid road condition to which the road condition corresponding to the first layered point cloud data belongs is determined, and the first inclination angle is an angle between a normal vector of the fitting plane of the first layered point cloud data and a horizontal plane on which the robot is located.

4. The method of claim 3, wherein, The type of the valid road condition at least includes a step, a dangerous slope and an ordinary slope; According to the first inclination angle of the fitting plane of the first layered point cloud data, the type of the valid road condition to which the road condition corresponding to the first layered point cloud data belongs is determined, including: If the first inclination angle is less than or equal to a first preset angle, it is determined that the road condition corresponding to the first layered point cloud data is the step; If the first inclination angle is greater than or equal to a second preset angle, it is determined that the road condition corresponding to the first layered point cloud data is the dangerous slope, and the second preset angle is greater than the first preset angle; If the first inclination angle is greater than the first preset angle and less than the second preset angle, it is determined that the road condition corresponding to the first layered point cloud data is the ordinary slope.

5. The method of claim 1, wherein, The multi-layered point cloud data is obtained by layering the falling point cloud data in a vertical direction, including The multi-layered point cloud data is obtained by layering the falling point cloud data in a vertical direction with a preset height as a unit, and a height of each layer of the multi-layered point cloud data in the vertical direction is the preset height.

6. The method according to any one of claims 1 to 5, wherein, Before the falling point cloud data is layered in the vertical direction, the method further includes: A second value is determined, and the second value is used to indicate a number of coordinate points in the falling point cloud data whose coordinate values in the vertical direction are less than a maximum height coordinate value; If the second value is less than a second preset number, a layerable point cloud data is determined from the falling point cloud data, the layerable point cloud data includes a set of coordinate points in the falling point cloud data whose coordinate values in the vertical direction are greater than or equal to the maximum height coordinate value; and the multi-layered point cloud data is obtained by layering the layerable point cloud data in the vertical direction. If the second value is greater than or equal to the second preset number, it is determined that the environment in front of the robot is a falling environment.

7. A robot fall environment determining apparatus characterized by comprising: The device comprises: A first determination module is configured to determine falling point cloud data from regional point cloud data, the coordinate point set included in the regional point cloud data being a set of three-dimensional coordinates corresponding to a plurality of pixel points in a field of view region of a camera carried by a robot in a robot coordinate system; A hierarchical module is configured to hierarchize the falling point cloud data in a vertical direction to obtain multi-layer hierarchical point cloud data; A second determination module is configured to determine road conditions corresponding to each layer of hierarchical point cloud data according to each layer of hierarchical point cloud data in the multi-layer hierarchical point cloud data; A third determination module is configured to determine whether the environment in front of the robot is a falling environment according to the road conditions corresponding to the multi-layer hierarchical point cloud data; The third determination module is further configured to: The road conditions corresponding to the multi-layer hierarchical point cloud data at least include a step and a dangerous slope; A first value is determined, the first value being used to indicate the number of layers of hierarchical point cloud data corresponding to the step in the multi-layer hierarchical point cloud data; If the first value is greater than or equal to a first number threshold, it is determined that the environment in front of the robot is the falling environment, the first number threshold being greater than 1; If the first value is equal to a second number threshold, it is determined that the environment in front of the robot is the falling environment in a case where a first step height corresponding to the step in the multi-layer hierarchical point cloud data and a second inclination angle of a fitting plane of hierarchical point cloud data corresponding to the step in the multi-layer hierarchical point cloud data satisfy a preset condition, the second number threshold being greater than 1 and less than the first number threshold; If the first value is equal to 1, it is determined that the environment in front of the robot is the falling environment in a case where the number of layers of hierarchical point cloud data corresponding to the dangerous slope in the multi-layer hierarchical point cloud data is greater than or equal to a preset number of layers; The third determination module is further configured to: The road conditions corresponding to the multi-layer hierarchical point cloud data further include an invalid road condition and an ordinary slope, if the first value is equal to 1, a second step height corresponding to the step in the multi-layer hierarchical point cloud data is determined in a case where other road conditions in the road conditions corresponding to the multi-layer hierarchical point cloud data except for the step are the invalid road condition; If the second step height is greater than or equal to a height threshold, it is determined that the environment in front of the robot is the falling environment; If the second step height is less than the height threshold, the robot is controlled to rotate by a third preset angle, and if it is determined according to regional point cloud data in a field of view region of a camera of the robot after rotation that the environment in front of the robot after rotation is free of the ordinary slope, it is determined that the environment in front of the robot before rotation is the falling environment.

8. A computer device, comprising: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is executed by the processor to implement the method according to any one of claims 1 to 6.

Images