Obstacle detection method and device for movable platform and movable platform

Abstract

A method and device (100) for detecting obstacles of a movable platform (200), and a movable platform (200), the method comprising: obtaining a point cloud corresponding to the surrounding environment of the movable platform (200) (S101); dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, the grid data including the point cloud density of each three-dimensional grid (S102); inputting the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform (200) (S103). A new three-dimensional point cloud representation method is proposed, which solves the problem that disordered point clouds cannot be used as input of neural networks by three-dimensionally gridding point clouds. In addition, irregular point clouds are processed into regular representation forms, which can better express the outline information of obstacles. At the same time, by using the grid data including the point cloud density as the input of the neural network, the number of point clouds of each three-dimensional grid can be distinguished, thereby improving the accuracy of obstacle detection.

Description

Technical Field

[0001] The present invention relates to the field of obstacle detection, and in particular to an obstacle detection method and device for a movable platform and a movable platform. Background Art

[0002] Unmanned vehicles use on-board sensors to sense the environment around the vehicle, and control the steering and speed of the vehicle based on the road, vehicle position and obstacle information obtained through perception, so that the vehicle can travel safely and reliably on the road. On-board sensors mainly include laser radar, millimeter wave radar and visual sensors. Among them, the basic principle of laser radar is to actively transmit laser pulse signals to the detected object, obtain the pulse signal reflected back, calculate the depth information of the detected object based on the time difference between the transmitted signal and the received signal; based on the known emission direction of the laser radar, obtain the angle information of the detected object relative to the laser radar; combine the above-mentioned depth and angle information to obtain a point cloud. However, the point cloud detected by the laser radar is unstructured and disordered data, which cannot be directly used as the input of the neural network. Therefore, how to use point clouds and neural networks to detect obstacles has become a hot issue that needs to be solved urgently. Summary of the invention

[0003] The present invention provides an obstacle detection method and device for a movable platform and a movable platform, which can process point clouds using a neural network to obtain obstacle information of the surrounding environment of the movable platform, and the obstacle detection has a high accuracy rate and can better describe the contour information of the obstacle.

[0004] Specifically, the present invention is achieved through the following technical solutions:

[0005] According to a first aspect of the present invention, there is provided a method for detecting an obstacle on a movable platform, the method comprising:

[0006] Acquire a point cloud corresponding to the surrounding environment of the movable platform;

[0007] Dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, wherein the grid data includes the point cloud density of each three-dimensional grid;

[0008] The grid data is input into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform.

[0009] According to a second aspect of the present invention, there is provided an obstacle detection device for a movable platform, comprising:

[0010] A storage device for storing program instructions;

[0011] One or more processors call program instructions stored in the storage device, and when the program instructions are executed, the one or more processors are individually or collectively configured to:

[0012] Acquire a point cloud corresponding to the surrounding environment of the movable platform;

[0013] Dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, wherein the grid data includes the point cloud density of each three-dimensional grid;

[0014] The grid data is input into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform.

[0015] According to a third aspect of the present invention, there is provided a movable platform, comprising:

[0016] Platform entity;

[0017] A power system, installed on the platform body, for providing power to the movable platform;

[0018] A sensor, installed on the platform body, for acquiring a point cloud corresponding to the surrounding environment of the movable platform;

[0019] And the obstacle detection device for the movable platform described in the second aspect.

[0020] It can be seen from the technical solutions provided by the above embodiments of the present invention that the present invention proposes a new type of three-dimensional point cloud representation method. By gridding the point cloud in three dimensions, the problem that disordered point clouds cannot be used as inputs of neural networks is solved. In addition, by processing irregular point clouds into regular representation forms, the contour information of obstacles can be better expressed. At the same time, by using grid data including point cloud density as input of the neural network, the number of point clouds of each three-dimensional grid can be distinguished, thereby improving the accuracy of obstacle detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 is a method flow chart of an obstacle detection method for a movable platform in one embodiment of the present invention;

[0023] Figure 2 is a flow chart of a method for upsampling a point cloud in one embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a specific network structure of a neural network in one embodiment of the present invention;

[0025] Figure 4 is a specific method flow chart of an obstacle detection method for a movable platform in one embodiment of the present invention;

[0026] Figure 5 is a structural block diagram of an obstacle detection device for a movable platform in one embodiment of the present invention;

[0027] Figure 6 It is a structural block diagram of a movable platform in one embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0030] The movable platform of the embodiment of the present invention can be a car, an unmanned aerial vehicle, a remote-controlled car, an unmanned ship or a robot, wherein the car can be an unmanned vehicle or a manned vehicle, and the unmanned aerial vehicle can be a drone or other unmanned aerial vehicles. Of course, the movable platform is not limited to the movable platforms listed above, and can also be other movable platforms.

[0031] Taking drone-driven vehicles as an example, unmanned vehicles use on-board sensors to sense the surrounding environment of the vehicle, and control the steering and speed of the vehicle based on the road, vehicle position and obstacle information obtained by the perception, so that the vehicle can travel safely and reliably on the road. On-board sensors mainly include laser radar, millimeter wave radar and visual sensor. Among them, the basic principle of laser radar is to actively transmit laser pulse signals to the detected object, obtain the pulse signal reflected back, calculate the depth information of the detected object according to the time difference between the transmitted signal and the received signal; based on the known emission direction of the laser radar, obtain the angle information of the detected object relative to the laser radar; and obtain the point cloud by combining the above-mentioned depth and angle information. However, the point cloud detected by the laser radar is unstructured disordered data and cannot be directly used as the input of the neural network. The present invention solves the problem that disordered point clouds cannot be used as the input of the neural network by three-dimensionally gridding the point cloud. In addition, the irregular point cloud is processed into a regular representation form, which can better express the contour information of the obstacle. At the same time, by using the grid data including the point cloud density as the input of the neural network, the number of point clouds of each three-dimensional grid can be distinguished, thereby improving the accuracy of obstacle detection.

[0032] Figure 1 FIG. 1 is a flow chart of a method for detecting an obstacle on a movable platform in one embodiment of the present invention; Figure 1 As shown, the obstacle detection method for a movable platform according to an embodiment of the present invention may include the following steps:

[0033] S101: Acquire a point cloud corresponding to the surrounding environment of the movable platform;

[0034] The method of acquiring the point cloud can be selected as needed. For example, in some embodiments, the point cloud is obtained by detection by a single sensor installed on a movable platform. The sensor can be a lidar, a millimeter-wave radar, etc. In addition, the point cloud corresponding to the surrounding environment can be acquired based on stereo vision.

[0035] In some embodiments, the point cloud is detected and determined by multiple sensors installed on a movable platform, and the multiple sensors can be a combination of at least two of a laser radar, a millimeter-wave radar, and a binocular camera; of course, the multiple sensors are not limited to the combination of the above sensors, but can also be a combination of at least two other sensors that can obtain a point cloud.

[0036] The point cloud obtained by using a single sensor is generally sparse. Under the condition of sparse point clouds, the obstacle detection effect is not good. Therefore, when obtaining the point cloud corresponding to the surrounding environment of the movable platform, the point cloud detected within the preset time length can be obtained, so as to achieve the accumulation of point clouds and achieve a basic improvement in the density of point clouds. Among them, the preset time length can be set as needed, such as 50ms or other time lengths. As a feasible implementation method, for the point cloud with a collection time of t, all point clouds obtained by lidar detection in the time period from t to t-50ms are obtained.

[0037] S102: Divide the point cloud into three-dimensional grids, and determine grid data corresponding to each three-dimensional grid, where the grid data includes the point cloud density of each three-dimensional grid;

[0038] In this step, grid division is performed in the x, y, and z directions at a certain resolution, so as to divide the point cloud in space into a three-dimensional grid, where x, y, and z can be a custom coordinate system or a world coordinate system. The resolution of the grid division in the x, y, and z directions can be equal or unequal.

[0039] The point cloud density of each three-dimensional grid is determined according to the number of point clouds of each three-dimensional grid and the volume of the corresponding three-dimensional grid. Optionally, the point cloud density of each three-dimensional grid is the ratio of the number of point clouds of each three-dimensional grid to the volume of the corresponding three-dimensional grid, that is, the point cloud density of each three-dimensional grid = the number of point clouds of the three-dimensional grid / the volume of the three-dimensional grid. The volume of each three-dimensional grid can be the same or different, and the method for calculating the point cloud density of each three-dimensional grid is not limited to the above calculation method, and other calculation methods can also be selected.

[0040] Furthermore, in some embodiments, the point cloud density of each three-dimensional grid is the point cloud density determined after normalizing the point cloud densities of all three-dimensional grids. The point cloud density determined after normalization is in a specific numerical range (usually greater than or equal to 0 and less than or equal to 1), which is convenient for subsequent data calculations. The point cloud density normalization method can be selected as needed. For example, in a feasible implementation method, the normalized point cloud density of each three-dimensional grid is the ratio of the point cloud density of each three-dimensional grid to the maximum value of the point cloud density of all three-dimensional grids, that is, the normalized point cloud density of each three-dimensional grid = the point cloud density of each three-dimensional grid / the maximum value of the point cloud density of all three-dimensional grids. Of course, the point cloud density normalization method can also adopt other normalization methods.

[0041] The grid data includes not only the above-mentioned point cloud density, but also other information, which is conducive to the subsequent further determination of obstacle information in the surrounding environment of the movable platform; optionally, the grid data may also include the position of each three-dimensional grid; optionally, the grid data may also include the size of each three-dimensional grid; optionally, the grid data may also include the position of each three-dimensional grid and the size of each three-dimensional grid.

[0042] The position of the three-dimensional grid can be represented by the vertex coordinates of the three-dimensional grid. For example, the shape of the three-dimensional grid can be a cube, and the position of the three-dimensional grid can be represented by the coordinates of the eight vertices of the three-dimensional grid. In addition, the position of the three-dimensional grid can also be represented by other methods, such as by the coordinates of the center point of the three-dimensional grid.

[0043] The point cloud obtained by a single sensor is generally sparse. Under the condition of sparse point cloud, the obstacle detection effect is not good. To solve the problem of sparse point cloud, related technologies use a combination of multiple sensors to obtain point cloud. For example, while using LiDAR to obtain point cloud, other sensors are used for auxiliary fusion to obtain dense and high-quality point cloud. The data fusion process between different sensors is relatively complicated. At the same time, multiple sensors also increase the hardware cost.

[0044] In this embodiment, the obstacle detection method for the movable platform may further include: after obtaining the point cloud corresponding to the surrounding environment of the movable platform, before dividing the point cloud into a three-dimensional grid, upsampling the point cloud, thereby obtaining a dense point cloud with high quality, solving the point cloud sparse problem. Further, in this embodiment, the up-sampled point cloud is divided into a three-dimensional grid.

[0045] Figure 2 It is a specific implementation method for upsampling point clouds, see Figure 2 , the process of upsampling the point cloud can include:

[0046] S201: for each point cloud, determining the number of point clouds in the first space within a preset distance from the point cloud;

[0047] In this embodiment, the point cloud in step S201 is the point cloud obtained by accumulating the point clouds in the above step S101.

[0048] Among them, the preset distance can be set according to needs, for example, it can be 1cm, 2cm, 3cm, 4cm, 5cm, 6cm, 7cm, 8cm, 9cm, 10cm, etc.

[0049] In this embodiment, the first space is a spherical space; it can be understood that the first space can also be a space region of a cube, a cuboid or other regular shapes formed with the point cloud as the center.

[0050] S202: If the number of point clouds in the first space is less than a first preset threshold, insert a new point cloud into the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

[0051] In this embodiment, if the number of point clouds in the first space is greater than or equal to the first preset threshold, there is no need to process the point clouds in the first space, thereby reducing the amount of calculation.

[0052] Through the above-mentioned upsampling processing, a point cloud with higher density is obtained, which realizes the purpose of achieving better obstacle detection effect under the condition of sparse point cloud; at the same time, the mobile platform can obtain better obstacle detection effect based on the point cloud obtained by detection of a single sensor, which reduces the requirement for the number of sensors and thus reduces the equipment cost.

[0053] The first preset threshold can be determined according to the missed detection rate and / or false detection rate when detecting obstacles. As a feasible implementation method, an initial threshold is set, and based on the initial threshold, the initial threshold is increased in sequence, and each threshold obtained after the increase is used as the first preset threshold in the above step S202, and the point cloud is upsampled, and obstacle detection is performed according to the point cloud after each upsampling process. If the gain corresponding to obstacle detection with the current threshold is less than the gain corresponding to obstacle detection with the previous threshold, the current threshold is used as the first preset threshold, and the gain is determined according to the missed detection rate and / or false detection rate when obstacle detection is performed each time. Even if a larger first preset threshold is selected, the gain corresponding to obstacle detection will not increase significantly, so there is no need to continue to increase the first preset threshold.

[0054] Optionally, the initial threshold is increased in sequence according to a preset step size, for example, the initial threshold is increased in sequence according to the rule of 1*preset length, 2*preset length, ..., N*preset step size, where N is the number of increasing processes; optionally, the preset step size is the initial threshold*0.2%; of course, the preset step size can also be set to other values.

[0055] The up-sampling processing method may be a bilinear interpolation method, or other interpolation methods may be used to insert a new point cloud into the first space.

[0056] S103: Input the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform.

[0057] The preset trained neural network may be a convolutional neural network, which is a feedforward neural network consisting of one or more convolutional layers and fully connected layers, as well as associated weights and pooling layers. This structure enables the convolutional neural network to utilize the two-dimensional structure of the input data.

[0058] Below, a specific network structure of a neural network is explained.

[0059] like Figure 3 As shown, the neural network of this embodiment may include a convolution layer and a pooling layer, a ROI Pooling layer, a fully connected layer and a correction layer.

[0060] Among them, the convolution layer and the pooling layer are used to process the grid data input into the neural network to obtain the first feature map. The convolution and pooling layers of this embodiment use conventional convolution and pooling operations to process the grid data to obtain the first feature map.

[0061] The ROI Pooling layer is used to predict the initial obstacle information of the surrounding environment of the movable platform based on the feature information of the first feature map. Optionally, the ROI Pooling layer predicts the position information of the initial obstacle candidate box, such as the position coordinates of the vertices of the initial obstacle candidate box or other information used to characterize the position of the initial obstacle candidate box.

[0062] The fully connected layer is used to process the first feature map processed by the ROI Pooling layer to obtain a second feature map. Optionally, the fully connected layer includes at least two, and the at least two fully connected layers sequentially process the first feature map processed by the ROI Pooling layer to obtain the second feature map. In this embodiment, the second feature map is obtained by sequentially processing the first feature map by two fully connected layers.

[0063] The correction layer is used to correct the initial obstacle information according to the feature information of the second feature map, and determine the obstacle information of the surrounding environment of the movable platform. In this embodiment, the correction layer can correct the position information of the initial obstacle candidate frame and / or screen the initial obstacle candidate frame according to the feature information of the second feature map to determine the final obstacle candidate frame. For example, the position information of the initial obstacle candidate frame predicted by the ROI Pooling layer may not be accurate. After the correction layer corrects the position information of the initial obstacle candidate frame, the accurate position information of the obstacle candidate frame can be obtained; for another example, the correction layer can exclude the candidate frame that is not an obstacle in the initial obstacle candidate frame predicted by the ROI Pooling layer.

[0064] The above obstacle information may include: the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame. Optionally, the position information of the obstacle candidate frame includes the position coordinates of the vertices of the obstacle candidate frame; optionally, the obstacle candidate frame is a cuboid or a cube, and the position information of the obstacle candidate frame includes the position coordinates of the 8 vertices of the cuboid or the cube; of course, the position information of the obstacle candidate frame may also include other information for indicating the position of the obstacle. Of course, the above obstacle information may also include other information of the obstacle candidate frame, such as the size of the obstacle candidate frame.

[0065] Furthermore, in certain embodiments, Figure 4 As shown, after the grid data is input into the pre-trained neural network to determine the obstacle information of the surrounding environment of the movable platform, the obstacle detection method of the movable platform may further include:

[0066] S401: Determine the prediction result of obstacles in the surrounding environment of the movable platform according to the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame.

[0067] Based on the above steps S101, S102, S103 and S401, the existence of obstacles in the surrounding environment of the movable platform can be accurately detected.

[0068] Among them, when determining the prediction result of the obstacle in the surrounding environment of the movable platform according to the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame, specifically, if the confidence of the obstacle candidate frame is greater than or equal to the preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate frame, and the position information of the obstacle candidate frame is used as the prediction result; if the confidence of the obstacle candidate frame is less than the preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate frame. The size of the above-mentioned preset confidence threshold can be set as needed. In this embodiment, the preset confidence threshold is greater than or equal to 80%, for example, the preset confidence threshold can be 80%, 85%, 90%, 95% or others.

[0069] It is understandable that the network structure of the neural network is not limited to Figure 3 The network structure shown can also be designed into other network structure forms.

[0070] It should be noted that in the embodiment of the present invention, when training the parameters of the neural network, the data format of the neural network to be trained is the same as the format of the grid data. The training process of the neural network may include but is not limited to the following steps:

[0071] (1) Annotate the input point cloud to obtain an annotated point cloud dataset, and use the point cloud dataset as training samples and test samples for the neural network;

[0072] (2) Construct the network structure of the above neural network;

[0073] (3) Upsampling and three-dimensional meshing are performed on each point cloud in the point cloud dataset and used as network input to train the neural network and generate parameters of the neural network.

[0074] The input point cloud is labeled, that is, obstacle information such as the position information and / or size of the obstacle is labeled according to the point cloud.

[0075] The method of upsampling each point cloud in the point cloud data set is the same as the method of upsampling the point cloud in the above embodiment, and the method of three-dimensionally meshing each point cloud in the point cloud data set is also the same as the method of three-dimensionally meshing the point cloud in the above embodiment (i.e., step S102).

[0076] The obstacle detection method for a movable platform of the embodiment of the present invention proposes a new three-dimensional point cloud representation method. By gridding the point cloud in three dimensions, the problem that disordered point clouds cannot be used as inputs of neural networks is solved. In addition, by processing irregular point clouds into regular representation forms, the contour information of obstacles can be better expressed. At the same time, by using grid data including point cloud density as input of the neural network, the number of point clouds of each three-dimensional grid can be distinguished, thereby improving the accuracy of obstacle detection.

[0077] In addition, it should be noted that the obstacle in this embodiment can be a static obstacle or a dynamic obstacle.

[0078] Corresponding to the obstacle detection method for a movable platform in the above embodiment, an embodiment of the present invention further provides an obstacle detection device for a movable platform, see Figure 5 The obstacle detection device 100 for the movable platform may include a storage device 110 and one or more processors 120 .

[0079] Among them, the storage device 110 is used to store program instructions; one or more processors 120 call the program instructions stored in the storage device 110. When the program instructions are executed, the one or more processors 120 are configured individually or collectively to: obtain a point cloud corresponding to the surrounding environment of the movable platform; divide the point cloud into a three-dimensional grid, and determine the grid data corresponding to each three-dimensional grid, the grid data includes the point cloud density of each three-dimensional grid; input the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment of the movable platform.

[0080] Optionally, the point cloud is obtained by detection by a laser radar installed on the movable platform.

[0081] Optionally, the one or more processors 120 are further configured, individually or collectively, to:

[0082] Obtain a point cloud corresponding to the surrounding environment of the movable platform within a preset time period.

[0083] Optionally, after acquiring the point cloud corresponding to the surrounding environment of the movable platform and before dividing the point cloud into a three-dimensional grid, the one or more processors 120 are configured individually or collectively to:

[0084] Performing upsampling processing on the point cloud;

[0085] When dividing the point cloud into a three-dimensional grid, the one or more processors 120 are individually or collectively configured to:

[0086] Divide the upsampled point cloud into a 3D grid.

[0087] Optionally, the one or more processors 120 are further configured, individually or collectively, to:

[0088] For each point cloud, determining the number of point clouds in the first space within a preset distance from the point cloud;

[0089] If the number of point clouds in the first space is less than a first preset threshold, a new point cloud is inserted into the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

[0090] Optionally, the upsampling processing method is a bilinear interpolation method.

[0091] Optionally, the point cloud density of each three-dimensional grid is determined according to the number of point clouds of each three-dimensional grid and the volume of the corresponding three-dimensional grid.

[0092] Optionally, the point cloud density of each three-dimensional grid is the ratio of the number of point clouds of each three-dimensional grid to the volume of the corresponding three-dimensional grid.

[0093] Optionally, the point cloud density of each three-dimensional grid is a point cloud density determined after normalizing the point cloud densities of all three-dimensional grids.

[0094] Optionally, the point cloud density determined after normalization is a ratio of the point cloud density of each three-dimensional grid to a maximum value of the point cloud densities of all three-dimensional grids.

[0095] Optionally, the grid data also includes the position of each three-dimensional grid and / or the size of each three-dimensional grid.

[0096] Optionally, the obstacle information includes: position information of the obstacle candidate frame and confidence of the obstacle candidate frame.

[0097] Optionally, the position information of the obstacle candidate box includes: position coordinates of vertices of the obstacle candidate box.

[0098] Optionally, the one or more processors 120 are individually or collectively configured to:

[0099] According to the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame, a prediction result of the obstacle in the surrounding environment of the movable platform is determined.

[0100] Optionally, the one or more processors are further configured, individually or collectively, to:

[0101] If the confidence of the obstacle candidate frame is greater than or equal to a preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate frame, and the position information of the obstacle candidate frame is used as the prediction result;

[0102] If the confidence of the obstacle candidate frame is less than a preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate frame.

[0103] Optionally, the neural network includes:

[0104] A convolution layer and a pooling layer, used to process the grid data input into the neural network to obtain a first feature map;

[0105] A ROI Pooling layer, used to predict initial obstacle information of the surrounding environment of the movable platform according to the feature information of the first feature map;

[0106] A fully connected layer, used for processing the first feature map processed by the ROI Pooling layer to obtain a second feature map;

[0107] The correction layer is used to correct the initial obstacle information according to the feature information of the second feature map, and determine the obstacle information of the surrounding environment of the movable platform.

[0108] The above-mentioned storage device may include a volatile memory (volatile memory), such as a random-access memory (RAM); the storage device may also include a non-volatile memory (non-volatile memory), such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the storage device 110 may also include a combination of the above-mentioned types of memory.

[0109] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0110] An embodiment of the present invention provides a movable platform, such as Figure 6 As shown, the movable platform 200 may include a platform body 210, a power system 220, a sensor 230 and an obstacle detection device 100 of the movable platform.

[0111] The power system 220 is installed on the platform body 210 and is used to provide power for the movable platform 200 .

[0112] The sensor 230 of this embodiment is installed on the platform body 210, and is used to obtain a point cloud corresponding to the surrounding environment of the movable platform 200. Optionally, the point cloud is obtained by detection by a single sensor, which may be a laser radar.

[0113] In this embodiment, the obstacle detection device 100 of the movable platform is electrically connected to the sensor 230. The obstacle detection device 100 of the movable platform of this embodiment is configured to: obtain a point cloud corresponding to the surrounding environment of the movable platform through the sensor 230; divide the point cloud into three-dimensional grids, and determine the grid data corresponding to each three-dimensional grid, the grid data including the point cloud density of each three-dimensional grid; input the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment of the movable platform 200.

[0114] The obstacle detection device 100 of the movable platform can be implemented as in the present invention. Figure 1 , Figure 2 as well as Figure 4 The obstacle detection method for the movable platform of the illustrated embodiment can be described with reference to the obstacle detection method for the movable platform of the above-mentioned embodiment to explain the movable platform 200 of this embodiment.

[0115] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the obstacle detection method for a movable platform of the above embodiment are implemented.

[0116] The computer-readable storage medium may be the internal storage unit of the pan-tilt head described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of the pan-tilt head, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the pan-tilt head and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the pan-tilt head, and may also be used to temporarily store data that has been output or is to be output.

[0117] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0118] The above disclosure is only part of the embodiments of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for detecting obstacles on a movable platform, characterized in that: The method comprises: Acquire a point cloud corresponding to the surrounding environment of the movable platform; Dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, the grid data including the point cloud density of each three-dimensional grid, wherein dividing the point cloud into three-dimensional grids includes performing grid division in three directions of x, y, and z according to a certain resolution, thereby dividing the point cloud in space into three-dimensional grids; The grid data is input into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform, wherein the pre-trained neural network includes a convolution and pooling layer, a ROIPooling layer, a fully connected layer and a correction layer, and the obstacle information includes the position coordinates of multiple vertices of a cube corresponding to an obstacle candidate box.

2. The method according to claim 1, characterized in that The point cloud is obtained by detecting a laser radar installed on the movable platform.

3. The method according to claim 1, characterized in that The obtaining of a point cloud corresponding to the surrounding environment of the movable platform includes: Obtain a point cloud corresponding to the surrounding environment of the movable platform within a preset time period.

4. The method according to claim 1, characterized in that: After obtaining the point cloud corresponding to the surrounding environment of the movable platform and before dividing the point cloud into three-dimensional grids, the method further includes: Performing upsampling processing on the point cloud; The dividing the point cloud into three-dimensional grids comprises: Divide the upsampled point cloud into a 3D grid.

5. The method according to claim 4, characterized in that The upsampling process on the point cloud comprises: For each point cloud, determining the number of point clouds in the first space within a preset distance from the point cloud; If the number of point clouds in the first space is less than a first preset threshold, a new point cloud is inserted into the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

6. The method according to claim 4, characterized in that The up-sampling processing method is a bilinear interpolation method.

7. The method according to claim 1, characterized in that The point cloud density of each three-dimensional grid is determined according to the number of point clouds of each three-dimensional grid and the volume of the corresponding three-dimensional grid.

8. The method according to claim 7, characterized in that The point cloud density of each three-dimensional grid is the ratio of the number of point clouds of each three-dimensional grid to the volume of the corresponding three-dimensional grid.

9. The method according to claim 1, characterized in that: The point cloud density of each three-dimensional grid is the point cloud density determined after normalizing the point cloud densities of all three-dimensional grids.

10. The method according to claim 9, characterized in that The point cloud density determined after normalization is a ratio of the point cloud density of each three-dimensional grid to a maximum value of the point cloud densities of all three-dimensional grids.

11. The method according to claim 1, characterized in that: The grid data also includes the position of each three-dimensional grid and / or the size of each three-dimensional grid.

12. The method according to claim 1, characterized in that The obstacle information includes: position information of the obstacle candidate frame and the confidence of the obstacle candidate frame.

13. The method according to claim 12, characterized in that The position information of the obstacle candidate frame includes: position coordinates of vertices of the obstacle candidate frame.

14. The method according to claim 12, characterized in that After inputting the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform, the method further includes: According to the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame, a prediction result of the obstacle in the surrounding environment of the movable platform is determined.

15. The method according to claim 14, characterized in that The step of determining the prediction result of the obstacle in the surrounding environment of the movable platform according to the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame includes: If the confidence of the obstacle candidate frame is greater than or equal to a preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate frame, and the position information of the obstacle candidate frame is used as the prediction result; If the confidence of the obstacle candidate frame is less than a preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate frame.

16. The method according to claim 1, characterized in that The convolution and pooling layer is used to process the grid data input into the neural network to obtain a first feature map; The ROIPooling layer is used to predict initial obstacle information of the surrounding environment of the movable platform according to the feature information of the first feature map; The fully connected layer is used to process the first feature map processed by the ROIPooling layer to obtain a second feature map; The correction layer is used to correct the initial obstacle information according to the feature information of the second feature map, and determine the obstacle information of the surrounding environment of the movable platform.

17. The method according to claim 16, characterized in that The fully connected layers include at least two.

18. An obstacle detection device for a movable platform, characterized in that: include: A storage device for storing program instructions; One or more processors call program instructions stored in the storage device, and when the program instructions are executed, the one or more processors are individually or collectively configured to: Acquire a point cloud corresponding to the surrounding environment of the movable platform; Dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, the grid data including the point cloud density of each three-dimensional grid, wherein dividing the point cloud into three-dimensional grids includes performing grid division in three directions of x, y, and z according to a certain resolution, thereby dividing the point cloud in space into three-dimensional grids; The grid data is input into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform, wherein the pre-trained neural network includes a convolution and pooling layer, a ROIPooling layer, a fully connected layer and a correction layer, and the obstacle information includes the position coordinates of multiple vertices of a cube corresponding to an obstacle candidate box.

19. The device according to claim 18, characterized in that The point cloud is obtained by detecting a laser radar installed on the movable platform.

20. The device according to claim 18, characterized in that The one or more processors are further configured, individually or collectively, to: Obtain a point cloud corresponding to the surrounding environment of the movable platform within a preset time period.

21. The device according to claim 18, characterized in that After acquiring the point cloud corresponding to the surrounding environment of the movable platform and before dividing the point cloud into a three-dimensional grid, the one or more processors are configured individually or collectively to: Performing upsampling processing on the point cloud; When dividing the point cloud into a three-dimensional grid, the one or more processors are individually or collectively configured to: Divide the upsampled point cloud into a 3D grid.

22. The device according to claim 21, characterized in that The one or more processors are further configured, individually or collectively, to: For each point cloud, determining the number of point clouds in the first space within a preset distance from the point cloud; If the number of point clouds in the first space is less than a first preset threshold, a new point cloud is inserted into the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

23. The device according to claim 21, characterized in that The up-sampling processing method is a bilinear interpolation method.

24. The device according to claim 18, characterized in that The point cloud density of each three-dimensional grid is determined according to the number of point clouds of each three-dimensional grid and the volume of the corresponding three-dimensional grid.

25. The device according to claim 24, characterized in that The point cloud density of each three-dimensional grid is the ratio of the number of point clouds of each three-dimensional grid to the volume of the corresponding three-dimensional grid.

26. The device according to claim 18, characterized in that The point cloud density of each three-dimensional grid is the point cloud density determined after normalizing the point cloud densities of all three-dimensional grids.

27. The device according to claim 26, characterized in that The point cloud density determined after normalization is a ratio of the point cloud density of each three-dimensional grid to a maximum value of the point cloud densities of all three-dimensional grids.

28. The device according to claim 18, characterized in that The grid data also includes the position of each three-dimensional grid and / or the size of each three-dimensional grid.

29. The device according to claim 18, characterized in that The obstacle information includes: position information of the obstacle candidate frame and the confidence of the obstacle candidate frame.

30. The device according to claim 29, characterized in that The position information of the obstacle candidate frame includes: position coordinates of vertices of the obstacle candidate frame.

31. The device according to claim 29, characterized in that After inputting the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment of the movable platform, the one or more processors are configured individually or collectively to: According to the position information of the obstacle candidate frame and the confidence of the obstacle candidate frame, a prediction result of the obstacle in the surrounding environment of the movable platform is determined.

32. The device according to claim 31, characterized in that The one or more processors are further configured, individually or collectively, to: If the confidence of the obstacle candidate frame is greater than or equal to a preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate frame, and the position information of the obstacle candidate frame is used as the prediction result; If the confidence of the obstacle candidate frame is less than a preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate frame.

33. The device according to claim 18, characterized in that The convolution layer and the pooling layer are used to process the grid data input into the neural network to obtain a first feature map; The ROIPooling layer is used to predict initial obstacle information of the surrounding environment of the movable platform according to the feature information of the first feature map; The fully connected layer is used to process the first feature map processed by the ROIPooling layer to obtain a second feature map; The correction layer is used to correct the initial obstacle information according to the feature information of the second feature map, and determine the obstacle information of the surrounding environment of the movable platform.

34. The device according to claim 33, characterized in that The fully connected layers include at least two.

35. A movable platform, characterized in that: include: Platform entity; A power system, installed on the platform body, for providing power to the movable platform; A sensor, mounted on the platform body, for acquiring a point cloud corresponding to the surrounding environment of the movable platform; and An obstacle detection device for a movable platform as described in any one of claims 18 to 34.

Images