Obstacle detection method and device for movable platform and movable platform

Abstract

An obstacle detection method and device (100) for a movable platform (200), and the movable platform (200). The method comprises: acquiring a point cloud corresponding to a surrounding environment where the movable platform (200) is located (S101); dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, the grid data comprising a point cloud density of each three-dimensional grid (S102); and inputting the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment where the movable platform (200) is located (S103). A novel three-dimensional point cloud representation mode is provided, the problem that disordered point clouds cannot serve as input of a neural network is solved through three-dimensional gridding of the point clouds, in addition, irregular point clouds are processed into a regular representation mode, and contour information of obstacles can be better represented. Meanwhile, by taking the grid data including the point cloud density as the input of the neural network, the number of the point clouds of each three-dimensional grid can be distinguished, and the accuracy of obstacle detection is improved.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201980011568.4. The filing date of the original application is June 28, 2019, and the invention title is "Obstacle Detection Method, Device and Mobile Platform for a Mobile Platform". Technical Field

[0002] The present invention relates to the field of obstacle detection, and particularly to an obstacle detection method, device and mobile platform for a mobile platform. Background Art

[0003] A driverless vehicle uses on-vehicle sensors to sense the surrounding environment of the vehicle, and controls the steering and speed of the vehicle according to the road, vehicle position and obstacle information obtained by the sensing, so that the vehicle can drive safely and reliably on the road. The on-vehicle sensors mainly include lidar, millimeter-wave radar and vision sensors. Among them, the basic principle of lidar is to actively emit laser pulse signals to the detected object and obtain the reflected pulse signals, calculate the depth information of the detected object according to the time difference between the emitted signal and the received signal; based on the known emission direction of the lidar, obtain the angle information of the detected object relative to the lidar; combine the aforementioned depth and angle information to obtain a point cloud. However, the point cloud detected by lidar is unstructured and disordered data, which cannot be directly used as the input of a neural network. Therefore, how to use the point cloud and the neural network to detect obstacles has become a hot issue to be solved urgently. Summary of the Invention

[0004] The present invention provides an obstacle detection method, device and mobile platform for a mobile platform, which can use a neural network to process the point cloud to obtain obstacle information of the surrounding environment where the mobile platform is located, and the accuracy of obstacle detection is high, and the contour information of the obstacle can be better described.

[0005] Specifically, the present invention is implemented through the following technical solutions: According to a first aspect of the present invention, there is provided an obstacle detection method for a mobile platform, the method comprising: Obtaining a point cloud corresponding to the surrounding environment where the mobile platform is located; Dividing the point cloud into three-dimensional grids and determining grid data corresponding to each three-dimensional grid, the grid data including the point cloud density of each three-dimensional grid; Inputting the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment where the mobile platform is located.

[0006] According to a second aspect of the present invention, there is provided an obstacle detection device for a mobile platform, comprising: A storage device for storing program instructions; One or more processors that invoke program instructions stored in the storage device, and when the program instructions are executed, the one or more processors are configured, individually or jointly, to: Obtain a point cloud corresponding to the surrounding environment where the mobile platform is located; Divide the point cloud into three-dimensional grids and determine grid data corresponding to each three-dimensional grid, where the grid data includes the point cloud density of each three-dimensional grid; Input the grid data into a pre-trained neural network to determine obstacle information in the surrounding environment where the mobile platform is located.

[0007] According to a third aspect of the present invention, there is provided a mobile platform, including: A platform main body; A power system installed on the platform main body for providing power to the mobile platform; A sensor installed on the platform main body for obtaining a point cloud corresponding to the surrounding environment where the mobile platform is located; And an obstacle detection device of the mobile platform described in the second aspect.

[0008] As can be seen from the technical solutions provided by the embodiments of the present invention above, the present invention proposes a new three-dimensional point cloud representation method. By three-dimensionally meshing the point cloud, the problem that unordered point clouds cannot be used as the input of a neural network is solved. In addition, processing irregular point clouds into a regular representation form can better represent the contour information of obstacles. At the same time, by using grid data including point cloud density as the input of the neural network, the number of point clouds in each three-dimensional grid can be distinguished, improving the accuracy of obstacle detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0010] Figure 1 is a flowchart of a method for detecting obstacles of a mobile platform in an embodiment of the present invention; Figure 2 is a flowchart of a method for upsampling a point cloud in an embodiment of the present invention; Figure 3 is a schematic diagram of a specific network structure of a neural network in an embodiment of the present invention; Figure 4It is a specific method flowchart of an obstacle detection method for a mobile platform in an embodiment of the present invention; Figure 5 It is a structural block diagram of an obstacle detection device for a mobile platform in an embodiment of the present invention; Figure 6 It is a structural block diagram of a mobile platform in an embodiment of the present invention. Detailed implementation manners

[0011] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0012] It should be noted that, without conflict, the features in the following embodiments and implementation manners can be combined with each other.

[0013] The mobile platform in the embodiment of the present invention can be an automobile, an unmanned aerial vehicle, a remote control vehicle, an unmanned ship or a robot. Among them, the automobile can be an autonomous vehicle or a manned vehicle, and the unmanned aerial vehicle can be a drone or other unmanned aerial vehicles. Of course, the mobile platform is not limited to the mobile platforms listed above, and can also be other mobile platforms.

[0014] Taking an autonomous vehicle as an example, an autonomous vehicle uses in-vehicle sensors to perceive the environment around the vehicle, and controls the steering and speed of the vehicle according to the road, vehicle position and obstacle information obtained by the perception, so that the vehicle can drive safely and reliably on the road. The in-vehicle sensors mainly include lidar, millimeter wave radar and vision sensors. Among them, the basic principle of lidar is to actively emit laser pulse signals to the detected object and obtain the reflected pulse signals, calculate the depth information of the detected object according to the time difference between the emitted signal and the received signal; based on the known emission direction of the lidar, obtain the angle information of the detected object relative to the lidar; combine the foregoing depth and angle information to obtain a point cloud. However, the point cloud detected by lidar is unstructured and disordered data, which cannot be directly used as the input of a neural network. The present invention solves the problem that the unordered point cloud cannot be used as the input of a neural network by three-dimensionally meshing the point cloud. In addition, processing the irregular point cloud into a regular representation form can better represent the contour information of the obstacle. At the same time, by using the grid data including the point cloud density as the input of the neural network, the number of point clouds in each three-dimensional grid can be distinguished, and the accuracy of obstacle detection can be improved.

[0015] Figure 1is a flowchart of a method for obstacle detection of a mobile platform in an embodiment of the present invention; as Figure 1 shown, the method for obstacle detection of the mobile platform in the embodiment of the present invention may include the following steps: S101: Obtain the point cloud corresponding to the surrounding environment where the mobile platform is located; The acquisition method of the point cloud can be selected according to needs. For example, in some embodiments, the point cloud is detected by a single sensor installed on the mobile platform. The sensor can be a lidar, a millimeter-wave radar, etc. In addition, the point cloud corresponding to the surrounding environment can also be obtained based on stereo vision.

[0016] In some embodiments, the point cloud is determined by detecting with multiple sensors installed on the mobile platform. The multiple sensors can be a combination of at least two of a lidar, a millimeter-wave radar, and a binocular camera; of course, the multiple sensors are not limited to the combination of the above sensors, and can also be a combination of at least two other sensors capable of obtaining point clouds.

[0017] The point cloud obtained by detecting with a single sensor is generally sparse, and under the condition of sparse point cloud, the obstacle detection effect is not good. Therefore, when obtaining the point cloud corresponding to the surrounding environment where the mobile platform is located, the point cloud detected within a preset time period can be obtained, so as to achieve the accumulation of the point cloud and realize the basic improvement of the point cloud density. Among them, the preset time period can be set according to needs, such as 50 ms or other time period sizes. As a feasible implementation method, for the point cloud at the acquisition moment t, all the point clouds detected by the lidar within the time period from t to t - 50 ms are obtained.

[0018] S102: Divide the point cloud into three-dimensional grids, and determine the grid data corresponding to each three-dimensional grid. The grid data includes the point cloud density of each three-dimensional grid; In this step, grid division is performed in the x, y, and z directions according to a certain resolution, so as to divide the point cloud in space into three-dimensional grids. Among them, x, y, and z can be a custom coordinate system or a world coordinate system. The resolution sizes of grid division in the x, y, and z directions can be equal or unequal.

[0019] Among them, the point cloud density of each three-dimensional grid is determined according to the number of point clouds in each three-dimensional grid and the volume of the corresponding three-dimensional grid. Optionally, the point cloud density of each three-dimensional grid is the ratio of the number of point clouds in each three-dimensional grid to the volume of the corresponding three-dimensional grid, that is, the point cloud density of each three-dimensional grid = the number of point clouds in this three-dimensional grid / the volume of this three-dimensional grid. Among them, the volumes of each three-dimensional grid can be the same or different, and the method for calculating the point cloud density of each three-dimensional grid is not limited to the above calculation method, and other calculation methods can also be selected.

[0020] Further, in some embodiments, the point cloud density of each three-dimensional grid is the point cloud density determined after normalizing the point cloud densities of all three-dimensional grids. The point cloud density determined after normalization is within a specific numerical range (usually greater than or equal to 0 and less than or equal to 1), which facilitates subsequent data operations. The method of normalizing the point cloud density can be selected as needed. For example, in a feasible implementation, the point cloud density of each three-dimensional grid after normalization is the ratio of the point cloud density of each three-dimensional grid to the maximum value of the point cloud densities of all three-dimensional grids. That is, the point cloud density of each three-dimensional grid after normalization = the point cloud density of each three-dimensional grid / the maximum value of the point cloud densities of all three-dimensional grids. Of course, other normalization methods can also be used for normalizing the point cloud density.

[0021] The grid data includes not only the above-mentioned point cloud density but also other information, which is conducive to further determining the obstacle information of the surrounding environment where the mobile platform is located in the subsequent process; optionally, the grid data can also include the position of each three-dimensional grid; optionally, the grid data can also include the size of each three-dimensional grid; optionally, the grid data can also include the position of each three-dimensional grid and the size of each three-dimensional grid.

[0022] Among them, the position of the three-dimensional grid can be represented by the vertex coordinates of the three-dimensional grid. By way of example, the shape of the three-dimensional grid can be a cube, and the position of the three-dimensional grid can be represented by the coordinates of the 8 vertices of the three-dimensional grid. In addition, the position of the three-dimensional grid can also be represented in other ways, such as by the coordinates of the center point of the three-dimensional grid.

[0023] The point cloud obtained by using a single sensor for detection is generally sparse. Under the condition of sparse point cloud, the obstacle detection effect is not good. To solve the problem of sparse point cloud, the related art uses a combination of multiple sensors to obtain the point cloud. For example, while using a lidar to detect and obtain the point cloud, other sensors are used for auxiliary fusion, so as to obtain a dense and high-quality point cloud. The data fusion process between different sensors is relatively complex. At the same time, multiple sensors also increase the hardware cost.

[0024] In this embodiment, the obstacle detection method of the mobile platform may further include: after obtaining the point cloud corresponding to the surrounding environment where the mobile platform is located and before dividing the point cloud into three-dimensional grids, performing upsampling processing on the point cloud, so as to obtain a dense and high-quality point cloud, thus solving the problem of sparse point cloud. Further, in this embodiment, the upsampled point cloud is divided into three-dimensional grids.

[0025] Figure 2 is a specific implementation of performing upsampling processing on the point cloud. Refer to Figure 2 The process of performing upsampling processing on the point cloud may include: S201: For each point cloud, determine the number of point clouds in the first space within a preset distance from the point cloud. In this embodiment, the point cloud in step S201 is the point cloud obtained by accumulating the point clouds in the above step S101.

[0026] Among them, the preset distance can be set as needed. For example, it can be 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, 6 cm, 7 cm, 8 cm, 9 cm, 10 cm, etc.

[0027] In this embodiment, the first space is a spherical space; it can be understood that the first space can also be a cubic, cuboid or other regularly shaped spatial region formed with this point cloud as the center.

[0028] S202: If the number of point clouds in the first space is less than the first preset threshold, insert new point clouds into the first space so that the number of point clouds in the first space reaches the first preset threshold after upsampling processing.

[0029] In this embodiment, if the number of point clouds in the first space is greater than or equal to the above first preset threshold, there is no need to process the point clouds in this first space, thereby reducing the computational amount.

[0030] Through the above upsampling processing, point clouds with higher density are obtained, achieving the purpose of achieving better obstacle detection effects under the condition of sparse point clouds; at the same time, the mobile platform can obtain better obstacle detection effects based on the point clouds detected by a single sensor, reducing the requirement for the number of sensors, thereby reducing the equipment cost.

[0031] The above first preset threshold can be determined according to the missed detection rate and / or false detection rate when detecting obstacles. As a feasible implementation method, set an initial threshold, on the basis of the initial threshold, increase the initial threshold in turn, and then use the obtained increased thresholds as the first preset threshold in the above step S202 to perform upsampling processing on the point clouds, and then perform obstacle detection according to the point clouds after each upsampling processing. If the gain corresponding to obstacle detection with the current threshold is less than the preset gain threshold compared with the gain corresponding to obstacle detection with the previous threshold, use the current threshold as the first preset threshold. The above gain is determined according to the missed detection rate and / or false detection rate when performing obstacle detection each time. Even if a larger first preset threshold is selected, there will be no obvious increase in the gain corresponding to obstacle detection, so there is no need to continue increasing the first preset threshold.

[0032] Optionally, the initial threshold is increased successively according to a preset step length. For example, the initial threshold is increased successively according to the rule of 1 * preset length, 2 * preset length,..., N * preset step length, where N is the number of times of the increase process; optionally, the preset step length is 0.2% of the initial threshold; of course, the preset step length can also be set to other values.

[0033] The upsampling process can be selected as the bilinear interpolation method, or other interpolation methods can be used to insert new point clouds in the first space.

[0034] S103: Input the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment where the mobile platform is located.

[0035] Among them, the pre-trained neural network can be a convolutional neural network. The convolutional neural network (Convolutional Neural Network, CNN) is a feedforward neural network. The convolutional neural network consists of one or more convolutional layers and fully connected layers, and also includes associated weights and pooling layers. This structure enables the convolutional neural network to utilize the two-dimensional structure of the input data.

[0036] Next, a specific network structure of a neural network will be described.

[0037] As Figure 3 shown, the neural network of this embodiment may include a convolutional layer, a pooling layer, an ROI Pooling layer, a fully connected layer, and a correction layer.

[0038] Among them, the convolutional layer and the pooling layer are used to process the grid data input into the neural network to obtain a first feature map. The convolutional and pooling layers of this embodiment use conventional convolutional and pooling operations to process the grid data to obtain a first feature map.

[0039] The ROI Pooling layer is used to predict the initial obstacle information of the surrounding environment where the mobile platform is located according to the feature information of the first feature map. Optionally, the ROI Pooling layer predicts the position information of the initial obstacle candidate box, such as the position coordinates of the vertices of the initial obstacle candidate box or other information used to characterize the position of the initial obstacle candidate box.

[0040] The fully connected layer is used to process the first feature map processed by the ROI Pooling layer to obtain a second feature map. Optionally, the fully connected layer includes at least two. The at least two fully connected layers process the first feature map processed by the ROI Pooling layer in sequence to obtain a second feature map. In this embodiment, the second feature map is obtained by two fully connected layers processing the first feature map in sequence.

[0041] The correction layer is used to correct the initial obstacle information according to the feature information of the second feature map, and determine the obstacle information of the surrounding environment where the mobile platform is located. In this embodiment, the correction layer can correct the position information of the initial obstacle candidate box and / or filter the initial obstacle candidate box according to the feature information of the second feature map to determine the final obstacle candidate box. For example, the position information of the initial obstacle candidate box predicted by the ROI Pooling layer may not be accurate enough. After the correction layer corrects the position information of the initial obstacle candidate box, the accurate position information of the obstacle candidate box can be obtained. Another example is that the correction layer can exclude the candidate boxes that are not obstacles in the initial obstacle candidate boxes predicted by the ROI Pooling layer.

[0042] The above-mentioned obstacle information may include: the position information of the obstacle candidate box and the confidence of the obstacle candidate box. Optionally, the position information of the obstacle candidate box includes the position coordinates of the vertices of the obstacle candidate box. Optionally, the obstacle candidate box is a cuboid or a cube, and the position information of the obstacle candidate box includes the position coordinates of the 8 vertices of the cuboid or cube. Of course, the position information of the obstacle candidate box may also include other information used to represent the position of the obstacle. Of course, the above-mentioned obstacle information may also include other information of the obstacle candidate box, such as the size of the obstacle candidate box.

[0043] Further, in some embodiments, as Figure 4 shown, after inputting the grid data into the pre-trained neural network to determine the obstacle information of the surrounding environment where the mobile platform is located, the obstacle detection method of the mobile platform may further include: S401: Determine the prediction result of the obstacles in the surrounding environment where the mobile platform is located according to the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

[0044] Based on the above steps S101, S102, S103 and S401, the presence of obstacles in the surrounding environment where the mobile platform is located can be accurately detected.

[0045] Among them, when determining the prediction result of the obstacles in the surrounding environment where the mobile platform is located according to the position information of the obstacle candidate box and the confidence of the obstacle candidate box, specifically, if the confidence of the obstacle candidate box is greater than or equal to the preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate box, and the position information of the obstacle candidate box is used as the prediction result; if the confidence of the obstacle candidate box is less than the preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate box. The size of the above-mentioned preset confidence threshold can be set as needed. In this embodiment, the preset confidence threshold is greater than or equal to 80%, for example, the preset confidence threshold can be 80%, 85%, 90%, 95% or others.

[0046] Understandably, the network structure of the neural network is not limited to Figure 3 the network structure shown, and can also be designed into other network structure forms.

[0047] It should be noted that in the embodiments of the present invention, when training the parameters of the above neural network, the data form input to the neural network to be trained is the same as the form of the above grid data. The training process of the above neural network may include but is not limited to the following steps: (1) Annotate the input point cloud to obtain an annotated point cloud data set, and use the point cloud data set as the training sample and test sample of the above neural network; (2) Construct the network structure of the above neural network; (3) Upsample and three-dimensionally grid each point cloud in the point cloud data set as the network input, train the neural network, and generate the parameters of the above neural network.

[0048] Among them, annotating the input point cloud means to annotate the position information of the obstacle and / or obstacle information such as size according to the point cloud.

[0049] The method of upsampling each point cloud in the point cloud data set is the same as the method of upsampling the point cloud in the above embodiment, and the method of three-dimensionally grid processing each point cloud in the point cloud data set is also the same as the method of three-dimensionally grid processing the point cloud in the above embodiment (i.e., step S102).

[0050] The obstacle detection method of the mobile platform in the embodiments of the present invention proposes a new three-dimensional point cloud representation method. By three-dimensionally gridifying the point cloud, the problem that unordered point clouds cannot be used as the input of the neural network is solved. In addition, processing irregular point clouds into regular representation forms can better represent the contour information of obstacles. At the same time, by using grid data including point cloud density as the input of the neural network, the number of point clouds in each three-dimensional grid can be distinguished, improving the accuracy of obstacle detection..

[0051] In addition, it should be noted that the obstacles in this embodiment can be static obstacles or dynamic obstacles.

[0052] Corresponding to the obstacle detection method of the mobile platform in the above embodiment, the embodiments of the present invention also provide an obstacle detection device for a mobile platform. Refer to Figure 5 , the obstacle detection device 100 of the mobile platform may include a storage device 110 and one or more processors 120.

[0053] Among them, a storage device 110 is used to store program instructions; one or more processors 120 call the program instructions stored in the storage device 110. When the program instructions are executed, the one or more processors 120 are individually or jointly configured to: obtain a point cloud corresponding to the surrounding environment where the movable platform is located; divide the point cloud into three-dimensional grids, and determine grid data corresponding to each three-dimensional grid, where the grid data includes the point cloud density of each three-dimensional grid; input the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment where the movable platform is located.

[0054] Optionally, the point cloud is detected by a lidar installed on the movable platform.

[0055] Optionally, the one or more processors 120 are individually or jointly further configured to: Obtain a point cloud corresponding to the surrounding environment where the movable platform is located within a preset time period.

[0056] Optionally, after obtaining the point cloud corresponding to the surrounding environment where the movable platform is located and before dividing the point cloud into three-dimensional grids, the one or more processors 120 are individually or jointly configured to: Perform upsampling processing on the point cloud; When dividing the point cloud into three-dimensional grids, the one or more processors 120 are individually or jointly configured specifically to: Divide the upsampled point cloud into three-dimensional grids.

[0057] Optionally, the one or more processors 120 are individually or jointly further configured to: For each point cloud, determine the number of point clouds in a first space within a preset distance from the point cloud; If the number of point clouds in the first space is less than a first preset threshold, insert new point clouds into the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

[0058] Optionally, the upsampling processing method is a bilinear interpolation method.

[0059] Optionally, the point cloud density of each three-dimensional grid is determined according to the number of point clouds in each three-dimensional grid and the volume of the corresponding three-dimensional grid.

[0060] Optionally, the point cloud density of each three-dimensional grid is the ratio of the number of point clouds in each three-dimensional grid to the volume of the corresponding three-dimensional grid.

[0061] Optionally, the point cloud density of each three-dimensional grid is the point cloud density determined after normalizing the point cloud densities of all three-dimensional grids.

[0062] Optionally, the point cloud density determined after normalization is the ratio of the point cloud density of each three-dimensional grid to the maximum value among the point cloud densities of all three-dimensional grids.

[0063] Optionally, the grid data further includes the position of each three-dimensional grid and / or the size of each three-dimensional grid.

[0064] Optionally, the obstacle information includes: the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

[0065] Optionally, the position information of the obstacle candidate box includes: the position coordinates of the vertices of the obstacle candidate box.

[0066] Optionally, the one or more processors 120 are individually or jointly configured to: Determine a prediction result of an obstacle in the surrounding environment of the mobile platform according to the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

[0067] Optionally, the one or more processors are further individually or jointly configured to: If the confidence of the obstacle candidate box is greater than or equal to a preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate box, and the position information of the obstacle candidate box is used as the prediction result; If the confidence of the obstacle candidate box is less than the preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate box.

[0068] Optionally, the neural network includes: A convolutional layer and a pooling layer for processing the grid data input to the neural network to obtain a first feature map; An ROI Pooling layer for predicting initial obstacle information in the surrounding environment of the mobile platform according to the feature information of the first feature map; A fully connected layer for processing the first feature map processed by the ROI Pooling layer to obtain a second feature map; A correction layer for correcting the initial obstacle information according to the feature information of the second feature map to determine the obstacle information in the surrounding environment of the mobile platform.

[0069] The above storage device may include a volatile memory, such as a random-access memory (RAM); the storage device may also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the storage device 110 may also include a combination of the above types of memories.

[0070] The above processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0071] An embodiment of the present invention provides a movable platform, such as Figure 6 As shown, the movable platform 200 may include a platform main body 210, a power system 220, a sensor 230, and an obstacle detection device 100 for the movable platform.

[0072] Among them, the power system 220 is installed on the platform main body 210 and is used to provide power for the movable platform 200.

[0073] The sensor 230 of this embodiment is installed on the platform main body 210 and is used to obtain a point cloud corresponding to the surrounding environment where the movable platform 200 is located. Optionally, the point cloud is detected by a single sensor, and the sensor may be a lidar.

[0074] In this embodiment, the obstacle detection device 100 for the movable platform is electrically connected to the sensor 230. The obstacle detection device 100 for the movable platform in this embodiment is configured to: obtain a point cloud corresponding to the surrounding environment where the movable platform is located through the sensor 230; divide the point cloud into three-dimensional grids and determine the grid data corresponding to each three-dimensional grid, where the grid data includes the point cloud density of each three-dimensional grid; input the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment where the movable platform 200 is located.

[0075] The obstacle detection device 100 of the mobile platform can implement the obstacle detection method of the mobile platform as described in the embodiments of the present invention Figure 1 、 Figure 2 and Figure 4 the obstacle detection method of the mobile platform shown in the embodiments. For the description of the mobile platform 200 in this embodiment, reference may be made to the obstacle detection method of the mobile platform in the above embodiments.

[0076] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the obstacle detection method of the mobile platform in the above embodiments are implemented.

[0077] The computer-readable storage medium may be an internal storage unit of the pan-tilt described in any of the foregoing embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of the pan-tilt, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. equipped on the device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the pan-tilt. The computer-readable storage medium is used to store the computer program and other programs and data required by the pan-tilt, and may also be used to temporarily store data that has been output or will be output.

[0078] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the embodiments of the above methods. Among them, the storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM), etc.

[0079] What is disclosed above is only some embodiments of the present invention. Of course, the scope of the rights of the present invention cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. A method for obstacle detection of a movable platform, characterized in that, The method includes: Obtaining a point cloud corresponding to the surrounding environment where the movable platform is located; Dividing the point cloud into three-dimensional grids, and determining grid data corresponding to each three-dimensional grid, where the grid data includes the point cloud density of each three-dimensional grid. Herein, dividing the point cloud into three-dimensional grids includes performing grid division in the x, y, and z directions respectively according to a certain resolution, so as to divide the point cloud in space into three-dimensional grids; Inputting the grid data into a pre-trained neural network to determine obstacle information of the surrounding environment where the movable platform is located, where the obstacle information includes the position coordinates of multiple vertices of a cube corresponding to an obstacle candidate box.

2. The method according to claim 1, wherein The point cloud is detected by a lidar installed on the movable platform.

3. The method according to claim 1, wherein The obtaining of the point cloud corresponding to the surrounding environment where the movable platform is located includes: Obtaining a point cloud corresponding to the surrounding environment where the movable platform is located within a preset time period.

4. The method according to claim 1, characterized in that, After obtaining the point cloud corresponding to the surrounding environment where the movable platform is located and before dividing the point cloud into three-dimensional grids, it further includes: Performing upsampling processing on the point cloud; The dividing of the point cloud into three-dimensional grids includes: Dividing the upsampled point cloud into three-dimensional grids.

5. The method according to claim 4, characterized in that, The performing of the upsampling processing on the point cloud includes: For each point cloud, determining the number of point clouds in a first space within a preset distance from the point cloud; If the number of point clouds in the first space is less than a first preset threshold, inserting new point clouds in the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

6. The method according to claim 4, characterized in that, The upsampling processing method is a bilinear interpolation method.

7. The method according to claim 1, characterized in that, The point cloud density of each three-dimensional grid is determined according to the number of point clouds in each three-dimensional grid and the volume of the corresponding three-dimensional grid.

8. The method according to claim 7, characterized in that The point cloud density of each three-dimensional grid is the ratio of the number of point clouds in each three-dimensional grid to the volume of the corresponding three-dimensional grid.

9. The method according to claim 1, wherein The point cloud density of each three-dimensional grid is the point cloud density determined after normalizing the point cloud densities of all three-dimensional grids.

10. The method according to claim 9, characterized in that The point cloud density determined after normalization is the ratio of the point cloud density of each three-dimensional grid to the maximum value among the point cloud densities of all three-dimensional grids.

11. The method according to claim 1, characterized in that, The grid data further includes the position of each three-dimensional grid and / or the size of each three-dimensional grid.

12. The method according to claim 1, characterized in that, The obstacle information includes: the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

13. The method according to claim 12, wherein The position information of the obstacle candidate box includes: the position coordinates of the vertices of the obstacle candidate box.

14. The method according to claim 12, wherein After inputting the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment where the movable platform is located, it further includes: Determining a prediction result of an obstacle in the surrounding environment where the movable platform is located according to the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

15. The method according to claim 14, wherein The determining of the prediction result of an obstacle in the surrounding environment where the movable platform is located according to the position information of the obstacle candidate box and the confidence of the obstacle candidate box includes: If the confidence level of the obstacle candidate box is greater than or equal to the preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate box, and the position information of the obstacle candidate box is used as the prediction result; If the confidence level of the obstacle candidate box is less than the preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate box.

16. The method according to claim 1, wherein The neural network includes: a convolution and pooling layer for processing the grid data input to the neural network to obtain a first feature map; An ROI Pooling layer for predicting the initial obstacle information of the surrounding environment where the mobile platform is located according to the feature information of the first feature map; A fully connected layer for processing the first feature map processed by the ROI Pooling layer to obtain a second feature map; A correction layer for correcting the initial obstacle information according to the feature information of the second feature map to determine the obstacle information of the surrounding environment where the mobile platform is located.

17. The method according to claim 16, characterized in that, The fully connected layer includes at least two.

18. An obstacle detection device for a movable platform, characterized in that, It includes: A storage device for storing program instructions; One or more processors that call the program instructions stored in the storage device, and when the program instructions are executed, the one or more processors are individually or jointly configured to: Obtain the point cloud corresponding to the surrounding environment where the mobile platform is located; Divide the point cloud into three-dimensional grids and determine the grid data corresponding to each three-dimensional grid. The grid data includes the point cloud density of each three-dimensional grid. Among them, dividing the point cloud into three-dimensional grids includes rasterizing in the x, y, and z directions respectively according to a certain resolution, so as to divide the point cloud in space into three-dimensional grids; Input the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment where the mobile platform is located. Among them, the obstacle information includes the position coordinates of multiple vertices of the cube corresponding to the obstacle candidate box.

19. The device according to claim 18, wherein The point cloud is detected by a lidar installed on the mobile platform.

20. The device according to claim 18, wherein, The one or more processors are further individually or jointly configured to: Obtain the point cloud corresponding to the surrounding environment where the mobile platform is located within a preset time period.

21. The device according to claim 18, characterized in that, After obtaining the point cloud corresponding to the surrounding environment where the mobile platform is located and before dividing the point cloud into three-dimensional grids, the one or more processors are individually or jointly configured to: Perform upsampling processing on the point cloud; When dividing the point cloud into three-dimensional grids, the one or more processors are specifically configured individually or jointly to: Divide the upsampled point cloud into three-dimensional grids.

22. The device according to claim 21, characterized in that, The one or more processors are further individually or jointly configured to: For each point cloud, determine the number of point clouds in the first space within a preset distance from the point cloud; If the number of point clouds in the first space is less than the first preset threshold, new point clouds are inserted into the first space so that the number of point clouds in the first space after upsampling processing reaches the first preset threshold.

23. The device according to claim 21, characterized in that, The upsampling processing method is a bilinear interpolation method.

24. The device according to claim 18, characterized in that, The point cloud density of each 3D grid is determined according to the number of the point cloud of each 3D grid and the volume of the corresponding 3D grid.

25. The device according to claim 24, characterized in that, The point cloud density of each 3D grid is the ratio of the number of the point cloud of each 3D grid to the volume of the corresponding 3D grid.

26. The device according to claim 18, characterized in that, The point cloud density of each 3D grid is the point cloud density determined after normalizing the point cloud densities of all 3D grids.

27. The device according to claim 26, characterized in that, The point cloud density determined after normalization is the ratio of the point cloud density of each 3D grid to the maximum value among the point cloud densities of all 3D grids.

28. The device according to claim 18, characterized in that, The grid data further includes the position of each 3D grid and / or the size of each 3D grid.

29. The device according to claim 18, wherein, The obstacle information includes: the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

30. The device according to claim 29, wherein The position information of the obstacle candidate box includes: the position coordinates of the vertices of the obstacle candidate box.

31. The device according to claim 29, wherein, After inputting the grid data into a pre-trained neural network to determine the obstacle information of the surrounding environment where the mobile platform is located, the one or more processors are individually or jointly configured to: Determine the prediction result of the obstacles in the surrounding environment where the mobile platform is located according to the position information of the obstacle candidate box and the confidence of the obstacle candidate box.

32. The device according to claim 31, characterized in that, The one or more processors are further individually or jointly configured to: If the confidence of the obstacle candidate box is greater than or equal to a preset confidence threshold, it is determined that there is an obstacle in the corresponding obstacle candidate box, and the position information of the obstacle candidate box is used as the prediction result; If the confidence of the obstacle candidate box is less than the preset confidence threshold, it is determined that there is no obstacle in the corresponding obstacle candidate box.

33. The device according to claim 18, wherein, The neural network includes: A convolutional layer and a pooling layer, which are used to process the grid data input into the neural network to obtain a first feature map; An ROI Pooling layer, which is used to predict the initial obstacle information of the surrounding environment where the mobile platform is located according to the feature information of the first feature map; A fully connected layer, which is used to process the first feature map processed by the ROI Pooling layer to obtain a second feature map; A correction layer, which is used to correct the initial obstacle information according to the feature information of the second feature map to determine the obstacle information of the surrounding environment where the mobile platform is located.

34. The device according to claim 33, wherein The fully connected layer includes at least two.

35. A movable platform, characterized in that, Comprising: A platform main body; A power system, installed on the platform main body, for providing power for the mobile platform; A sensor, installed on the platform main body, for acquiring the point cloud corresponding to the surrounding environment where the mobile platform is located; and The obstacle detection device of the mobile platform according to any one of claims 18-34.

Images