A roadblock detection method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a roadblock detection method and device, electronic equipment and storage medium. The method comprises: performing target detection on a real scene image collected by an image collection component to obtain a 2D detection result of a roadblock; obtaining point cloud cluster information of the roadblock based on the 2D detection result and point cloud information corresponding to the real scene image; and obtaining a detection result of the roadblock based on the point cloud cluster information and a preset roadblock shape, wherein the detection result of the roadblock comprises at least one of roadblock position information, roadblock volume information and roadblock orientation information. According to the present disclosure, the detection result of the roadblock is obtained based on the point cloud cluster information of the roadblock and the preset roadblock shape, and the accuracy of roadblock detection can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and more specifically, to a road obstacle detection method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] As driverless cars become a part of people's lives, autonomous driving technology has gradually become a key research area in the automotive field. Since road obstacles are an important component in real-world road scenarios, detecting and avoiding road obstacles during driving is also an issue that cannot be ignored in autonomous driving technology.

[0003] In existing technologies, roadblocks are typically detected using vehicle sensing modules (such as radar). However, due to differences in the shape, size, material, and placement of various roadblocks, the detection results often differ from the actual results, thus affecting driving safety. Therefore, improving the accuracy of roadblock detection has become an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides a road obstacle detection method, including:

[0005] Target detection is performed on the real-world scene images acquired by the image acquisition unit to obtain 2D detection results of road obstacles;

[0006] Based on the 2D detection results and the point cloud information corresponding to the real scene image, the point cloud cluster information of the roadblock is obtained;

[0007] Based on the point cloud cluster information and the preset roadblock shape, the detection result of the roadblock is obtained. The detection result of the roadblock includes at least one of the roadblock location information, roadblock volume information and roadblock orientation information.

[0008] In this embodiment of the disclosure, the point cloud cluster information of the roadblock is obtained by using the 2D detection results and the point cloud information corresponding to the real scene image. Combined with the preset roadblock shape, the location information, volume information and orientation information of the roadblock are obtained. In this way, the accuracy of roadblock detection can be improved.

[0009] In one optional implementation, obtaining the point cloud cluster information of the roadblock based on the 2D detection results and the point cloud information corresponding to the real-world scene image includes:

[0010] Based on preset map information, the point cloud information is segmented to obtain target point cloud information;

[0011] Based on the 2D detection results and the target point cloud information, the point cloud cluster information of the roadblock is obtained.

[0012] In this embodiment of the disclosure, the point cloud information is segmented based on preset map information, which can reduce the range of the point cloud information. Furthermore, the range of the target point cloud information can be further reduced through 2D detection results, which is beneficial to improving the accuracy of determining the point cloud cluster information of roadblocks.

[0013] In one optional implementation, the 2D detection result includes the display boundary of the roadblock in the real-world scene image; obtaining the point cloud cluster information of the roadblock based on the 2D detection result and the target point cloud information includes:

[0014] Determine whether each point in the target point cloud information falls within the display boundary range of the roadblock;

[0015] Points in the target point cloud information that fall within the display boundary range of the roadblock are identified as the point cloud cluster information of the roadblock.

[0016] In this embodiment of the disclosure, the point cloud cluster information of the roadblock is obtained by using the display boundary of the roadblock in the real scene image, which can improve the accuracy of determining the point cloud cluster information of the roadblock.

[0017] In one optional implementation, determining whether each point in the target point cloud information falls within the display boundary range of the roadblock includes:

[0018] Based on the absolute pose of the radar device collecting point cloud data, the relative pose of the image acquisition component with respect to the radar device, and the internal parameters of the image acquisition component, it is determined whether each point in the target point cloud information falls within the display boundary range of the roadblock.

[0019] In this embodiment of the disclosure, the coordinates of a point in the target point cloud information in the image coordinate system are determined by the transformation relationship of the coordinate system, and the point is judged to be within the display boundary of the roadblock based on the transformed coordinates, which can improve the efficiency and accuracy of the judgment.

[0020] In one optional implementation, obtaining the detection result of the roadblock based on the point cloud cluster information and a preset roadblock shape includes:

[0021] The location information of the roadblock is determined based on the average depth information of the point cloud cluster information;

[0022] Based on the point cloud cluster information and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0023] In this embodiment of the disclosure, the location information of the roadblock can be determined based on the average depth information, and the volume information and / or the orientation information of the roadblock can be determined based on the point cloud cluster information and the preset roadblock shape. In this way, the determination of multiple dimensions of information of the roadblock is realized.

[0024] In one optional implementation, the preset roadblock shape is the roadblock shape under a preset viewpoint; determining the roadblock volume information and / or the roadblock orientation information based on the point cloud cluster information and the preset roadblock shape includes:

[0025] Determine the point cloud cluster information in the point cloud cluster image under the preset viewpoint;

[0026] Based on the point cloud cluster image and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0027] In this embodiment of the disclosure, since the preset roadblock shape and the point cloud cluster image are both roadblock views from the same preset perspective, the accuracy of the determined roadblock volume information and / or the roadblock orientation information can be improved based on the point cloud cluster image and the preset roadblock shape.

[0028] In one optional implementation, determining the roadblock volume information and / or the roadblock orientation information based on the point cloud cluster image and the preset roadblock shape includes:

[0029] Determine the center point of the point cloud cluster image, and set the center point of the point cloud cluster image as the center point of the preset roadblock shape;

[0030] The preset roadblock shape is rotated multiple times around the center point, and the overlap area between the preset roadblock shape and the point cloud cluster image after each rotation is determined.

[0031] Based on the overlapping area between the preset roadblock shape after each rotation and the point cloud cluster image, the roadblock volume information and / or the roadblock orientation information are determined.

[0032] In this embodiment of the disclosure, by rotating the preset roadblock shape around the center point multiple times, a smaller granular overlapping area can be obtained; secondly, based on the overlapping area of ​​the preset roadblock shape and the point cloud cluster image after each rotation, the shape and orientation of the roadblock can be accurately determined, and it conforms to the shape and orientation of roadblocks in real-world scenarios, thereby improving the accuracy of roadblock detection.

[0033] In one optional implementation, determining the obstacle volume information and / or the obstacle orientation information based on the overlapping area of ​​the preset obstacle shape and the point cloud cluster image after each rotation includes:

[0034] When the overlap area between the preset roadblock shape and the point cloud cluster image is maximized, the orientation information of the preset roadblock shape is determined as the roadblock orientation information; and / or,

[0035] When the area of ​​overlap between the preset roadblock shape and the point cloud cluster image is the largest, the volume of the point cloud cluster information corresponding to the largest overlap area is determined as the roadblock volume information.

[0036] In this embodiment of the disclosure, the preset roadblock shape has the largest overlap area with the point cloud cluster image. That is, the orientation of the preset roadblock shape is more consistent with the orientation of roadblocks in the real scene, and the orientation information of the roadblock determined by the orientation information of the preset roadblock shape is more accurate. Secondly, the volume corresponding to the maximum overlap area is more consistent with the volume of roadblocks in the real scene. Determining the volume corresponding to the maximum overlap area as the roadblock volume can improve the accuracy of roadblock detection.

[0037] This disclosure also provides a road obstacle detection device, the device comprising:

[0038] The target detection module is used to perform target detection on the real-world scene images acquired by the image acquisition unit to obtain 2D detection results of road obstacles;

[0039] The point cloud processing module is used to obtain the point cloud cluster information of the roadblock based on the 2D detection results and the point cloud information corresponding to the real scene image;

[0040] The result determination module is used to obtain the detection result of the road obstacle based on the point cloud cluster information and the preset road obstacle shape. The detection result of the road obstacle includes at least one of road obstacle location information, road obstacle volume information and road obstacle orientation information.

[0041] In one optional implementation, the point cloud processing module is specifically used for:

[0042] Based on preset map information, the point cloud information is segmented to obtain target point cloud information;

[0043] Based on the 2D detection results and the target point cloud information, the point cloud cluster information of the roadblock is obtained.

[0044] In one optional implementation, the 2D detection result includes the display boundary of the obstacle in the real-world scene image; the point cloud processing module is specifically used for:

[0045] Determine whether each point in the target point cloud information falls within the display boundary range of the roadblock;

[0046] Points in the target point cloud information that fall within the display boundary range of the roadblock are identified as the point cloud cluster information of the roadblock.

[0047] In one optional implementation, the point cloud processing module is specifically used for:

[0048] Based on the absolute pose of the radar device collecting point cloud data, the relative pose of the image acquisition component with respect to the radar device, and the internal parameters of the image acquisition component, it is determined whether each point in the target point cloud information falls within the display boundary range of the roadblock.

[0049] In one optional implementation, the result determination module is specifically used for:

[0050] The location information of the roadblock is determined based on the average depth information of the point cloud cluster information;

[0051] Based on the point cloud cluster information and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0052] In one optional implementation, the preset roadblock shape is the roadblock shape under a preset viewpoint; the result determination module is specifically used for:

[0053] Determine the point cloud cluster information in the point cloud cluster image under the preset viewpoint;

[0054] Based on the point cloud cluster image and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0055] In one optional implementation, the result determination module is specifically used for:

[0056] Determine the center point of the point cloud cluster image, and set the center point of the point cloud cluster image as the center point of the preset roadblock shape;

[0057] The preset roadblock shape is rotated multiple times around the center point, and the overlap area between the preset roadblock shape and the point cloud cluster image after each rotation is determined.

[0058] Based on the overlapping area between the preset roadblock shape after each rotation and the point cloud cluster image, the roadblock volume information and / or the roadblock orientation information are determined.

[0059] In one optional implementation, the result determination module is specifically used for:

[0060] When the overlap area between the preset roadblock shape and the point cloud cluster image is maximized, the orientation information of the preset roadblock shape is determined as the roadblock orientation information; and / or,

[0061] When the area of ​​overlap between the preset roadblock shape and the point cloud cluster image is the largest, the volume of the point cloud cluster information corresponding to the largest overlap area is determined as the roadblock volume information.

[0062] This disclosure also provides an electronic device, including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the roadblock detection method described above are performed.

[0063] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the roadblock detection method described above.

[0064] For a description of the effects of the aforementioned obstacle detection device, electronic equipment, and computer-readable storage medium, please refer to the description of the obstacle detection method above; it will not be repeated here.

[0065] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of a road scene provided in an embodiment of the present disclosure;

[0068] Figure 2 A flowchart of a road obstacle detection method provided in an embodiment of this disclosure;

[0069] Figure 3 This is a schematic diagram of the 2D detection results provided in the embodiments of this disclosure;

[0070] Figure 4A flowchart illustrating a method for obtaining point cloud cluster information of roadblocks provided in an embodiment of this disclosure;

[0071] Figure 5 A detailed flowchart of a method for obtaining point cloud cluster information of roadblocks provided in an embodiment of this disclosure;

[0072] Figure 6 A flowchart illustrating a method for determining the detection result of a road obstacle, provided in an embodiment of this disclosure;

[0073] Figure 7 A flowchart illustrating a method for determining obstacle volume information and / or obstacle orientation information provided in an embodiment of this disclosure;

[0074] Figure 8 This is a schematic diagram of a point cloud cluster image and a preset roadblock shape provided in an embodiment of this disclosure;

[0075] Figure 9 A detailed flowchart of a method for determining obstacle volume information and / or obstacle orientation information provided in an embodiment of this disclosure;

[0076] Figure 10 This is a schematic diagram of the structure of a road obstacle detection device provided in an embodiment of the present disclosure;

[0077] Figure 11 This is a schematic diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0079] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0080] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0081] As driverless cars become a part of people's lives, autonomous driving technology has gradually become a key research area in the automotive field. Since road obstacles are an important component in real-world road scenarios, detecting and avoiding road obstacles during driving has become an issue that cannot be ignored in autonomous driving technology.

[0082] Please see Figure 1 , Figure 1 This is a schematic diagram of a road scene provided as an embodiment of this disclosure. Figure 1 As shown in the diagram, the road includes an unmanned vehicle 200 and roadblocks 300. The unmanned vehicle 200 is equipped with a camera 400 and an onboard radar 500. During the operation of the unmanned vehicle 200, the camera 400 captures 2D scene images of the current road in real time to detect whether there are roadblocks. At the same time, the onboard radar 500 acquires 3D point cloud information of the current road scene in real time. Thus, roadblocks in the road can be detected and avoided based on both the 2D scene images and the 3D point cloud information.

[0083] In existing technologies, roadblocks are typically detected using vehicle sensing modules (such as radar). However, due to differences in the shape, size, material, and placement of various roadblocks, the detection results often differ from the actual results, thus affecting driving safety. Therefore, improving the accuracy of roadblock detection has become an urgent problem to be solved.

[0084] Based on the above research, this disclosure provides a roadblock detection method, which includes: performing target detection on a real-world scene image acquired by an image acquisition unit to obtain a 2D detection result of the roadblock; obtaining point cloud cluster information of the roadblock based on the 2D detection result and point cloud information corresponding to the real-world scene image; and obtaining a detection result of the roadblock based on the point cloud cluster information and a preset roadblock shape, wherein the detection result of the roadblock includes at least one of roadblock location information, roadblock volume information, and roadblock orientation information.

[0085] In this embodiment, the point cloud cluster information of the roadblock is obtained based on the point cloud information corresponding to the real scene image through 2D detection results. Combined with the preset roadblock shape, the location information, volume information and orientation information of the roadblock are obtained. In this way, the accuracy of roadblock detection can be improved.

[0086] To facilitate understanding of this embodiment, a detailed description of the road obstacle detection method disclosed in this disclosure is provided first. The execution entity of the road obstacle detection method provided in this disclosure is generally an electronic device with a certain computing power. This electronic device may include, for example, a terminal device, a server, or other processing devices. The terminal device may include mobile phones, tablets, in-vehicle devices, and wearable devices. The server may be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. Other processing devices may include devices including processors and memory, and are not limited here. In some possible implementations, the road obstacle detection method can be implemented by the processor calling computer-readable instructions stored in memory.

[0087] Please see Figure 2 , Figure 2 This is a flowchart illustrating a road obstacle detection method provided in an embodiment of this disclosure. Figure 2 As shown, the obstacle detection method provided in this embodiment includes the following steps S101 to S103:

[0088] S101, target detection is performed on the real-world scene image acquired by the image acquisition unit to obtain the 2D detection result of the road obstacle.

[0089] The image acquisition component can be a vehicle-mounted camera; for example, multiple cameras can be installed on the exterior of the vehicle (e.g., ...). Figure 1 The camera 400 shown can specifically be multiple monocular cameras. The real-world scene image can be a road scene image captured by a vehicle-mounted camera.

[0090] Target detection, also known as target extraction, is used to extract target objects in complex scene images. In some implementations, target detection of real-world scene images acquired by the image acquisition unit can be achieved through neural network technology.

[0091] Please see Figure 3 , Figure 3 This is a schematic diagram of the 2D detection results provided in an embodiment of this disclosure. Figure 3As shown, the 2D detection result includes the display boundary of the roadblock in the real-world scene image. It can be understood that the display boundary is the outline of the roadblock in the 2D detection result. In this embodiment, the bounding rectangle 600 of the roadblock 300 is the 2D detection result.

[0092] Specifically, a trained neural network model (e.g., Faster R-CNN, SSD, or YOLO neural network model) can be obtained by training some real-world scene image samples (e.g., real-world scene images containing roadblocks). When performing object detection, the real-world scene image samples can be input into the trained neural network model to detect whether there is a 2D detection result of roadblocks in the real-world scene image sample.

[0093] S102, based on the 2D detection results and the point cloud information corresponding to the real scene image, the point cloud cluster information of the roadblock is obtained.

[0094] For example, the point cloud information refers to a massive set of points representing the surface characteristics of a target object, which can be obtained by vehicle-mounted radar from the radar of objects in a real scene. The point cloud information may include: orientation information, distance information, etc.

[0095] Among them, the vehicle-mounted radar can be a vehicle-mounted lidar. Specifically, vehicle-mounted lidar includes single-beam lidar and multi-beam lidar. In this embodiment, the vehicle-mounted lidar used is a 64-beam lidar.

[0096] It is understood that since the point cloud information corresponding to the real scene image includes point cloud information of different objects, such as point cloud information of roads, point cloud information of roadblocks, and point cloud information of road medians, the point cloud cluster information corresponding to the roadblock can be obtained from the point cloud information through the 2D detection results.

[0097] S103, based on the point cloud cluster information and the preset roadblock shape, obtain the detection result of the roadblock, the detection result of the roadblock includes at least one of roadblock location information, roadblock volume information and roadblock orientation information.

[0098] The preset roadblock shape refers to a roadblock shape predetermined based on empirical values. In some implementations, considering the shape of roadblocks in real-world scenarios, the preset roadblock shape is usually a flat and elongated shape.

[0099] It is understandable that since the point cloud cluster information includes information such as the location or distance of the roadblock, the location information of the roadblock can be determined based on the point cloud cluster information.

[0100] In real-world scenarios, roadblocks are typically three-dimensional objects with length, width, and height. Since the volume and orientation of roadblocks are not fixed, it is necessary to determine the volume and orientation information of such roadblocks. That is, based on the point cloud cluster information and the preset roadblock shape, the roadblock volume information and / or roadblock orientation information are determined so that vehicles can accurately avoid obstacles.

[0101] In this embodiment of the disclosure, during the roadblock detection process, the point cloud cluster information of the roadblock is obtained based on the point cloud information corresponding to the real scene image through 2D detection results. Combined with the preset roadblock shape, the location information, volume information and orientation information of the roadblock are obtained. In this way, the accuracy of roadblock detection can be improved.

[0102] In some implementations, regarding step S102 above, when obtaining the point cloud cluster information of the roadblock based on the 2D detection results and the point cloud information corresponding to the real-world scene image, such as... Figure 4 As shown, it may include the following S1021 to S1022:

[0103] S1021, Based on preset map information, the point cloud information is segmented to obtain target point cloud information.

[0104] For example, point cloud information can be segmented based on pre-stored map information to extract point cloud information of different objects. In this embodiment, in order to improve the segmentation accuracy, a high-precision map is used. A high-precision map refers to a map with high precision, high dynamics, and multiple dimensions, that is, a map with centimeter-level precision, good real-time performance, and detailed lane information and detailed road attribute information.

[0105] In some implementations, random sampling consensus (RanSac), point cloud segmentation techniques based on proximity information, point cloud frequency-based segmentation (filtering) techniques, minimum cut techniques, or super-volume clustering segmentation techniques can be used, and no specific limitation is made here.

[0106] Specifically, since the point cloud information of different objects carries different orientation and height information—for example, the point cloud information of the ground and the point cloud information of a road obstacle carry different height information—the point cloud information can be segmented to obtain target point cloud information. In this embodiment, target point cloud information refers to point cloud information whose height is the same as that of the road obstacle.

[0107] S1022, Based on the 2D detection results and the target point cloud information, the point cloud cluster information of the roadblock is obtained.

[0108] Since the 2D detection results include roadblocks and a small number of other objects, by matching the bounding rectangle of the roadblock with the target point cloud information, that is, by using the 2D detection results, the range of the target point cloud information can be narrowed down, and thus the point cloud cluster information of the roadblock can be obtained.

[0109] Regarding step S1022 above, when obtaining the point cloud cluster information of the roadblock based on the 2D detection results and the target point cloud information, as follows: Figure 5 As shown, it may include the following S10221 to S10222:

[0110] S10221, determine whether each point in the target point cloud information falls within the display boundary range of the roadblock.

[0111] Since the target point cloud information is point cloud information with the same height as the roadblock, it may contain point clouds of other objects besides the roadblock point cloud. Therefore, it is necessary to determine whether each point in the target point cloud information falls within the display boundary range of the roadblock in order to further determine the roadblock point cloud information in the target point cloud information.

[0112] Specifically, based on the absolute pose of the radar device that collects point cloud data, the relative pose of the image acquisition component with respect to the radar device, and the internal parameters of the image acquisition component, it can be determined whether each point in the target point cloud information falls within the display boundary range of the roadblock.

[0113] The absolute pose of the radar device that collects point cloud data is the absolute pose of the radar device relative to the world coordinate system.

[0114] For example, to improve the accuracy of determining whether each point in the target point cloud information falls within the display boundary range of the roadblock, the first coordinate of each point in the target point cloud information in the world coordinate system can be converted into a second coordinate in the visual camera coordinate system based on the absolute pose of the radar device relative to the world coordinate system and the relative pose of the image acquisition component relative to the radar device; and based on the internal parameters of the image acquisition component, the second coordinate is converted into a third coordinate in the image coordinate system; according to the third coordinate and the coordinate range of the display boundary of the roadblock in the real scene image in the image coordinate system, it is determined whether each point in the target point cloud information falls within the display boundary range of the roadblock in the real scene image. That is, the coordinates of the points in the target point cloud information in the image coordinate system can be determined through the transformation relationship between coordinate systems, and the transformed coordinates can be used to determine whether each point in the target point cloud information falls within the display boundary range of the roadblock in the real scene image.

[0115] S10222, the points in the target point cloud information that fall within the display boundary range of the roadblock are determined as the point cloud cluster information of the roadblock.

[0116] It can be understood that the target point cloud information falling within the display boundary of the roadblock is the point cloud cluster information of the roadblock.

[0117] In some implementations, regarding step S103 above, when obtaining the detection result of the roadblock based on the point cloud cluster information and the preset roadblock shape, as follows: Figure 6 As shown, it may include the following S1031 to S1032:

[0118] S1031, Based on the average depth information of the point cloud cluster information, determine the location information of the roadblock.

[0119] The average depth information is the distance information between the roadblock and the vehicle-mounted radar. It can be understood that the location information of the roadblock can be determined based on the distance information.

[0120] S1032, Based on the point cloud cluster information and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0121] Since the point cloud cluster information is 3D information, the volume information of the roadblock and / or the orientation information of the roadblock can be determined based on the point cloud cluster information and the preset roadblock shape.

[0122] In some implementations, for step S1032 above, the preset roadblock shape is the roadblock shape under a preset viewpoint; when determining the roadblock volume information and / or the roadblock orientation information based on the point cloud cluster information and the preset roadblock shape, such as Figure 7 As shown, it may include the following S10321~S10322:

[0123] S10321, Determine the point cloud cluster information as a point cloud cluster image under the preset viewpoint.

[0124] In some implementations, the preset viewpoint is a top-down viewpoint with the ground as the reference plane.

[0125] The point cloud cluster image can be a projection image of the point cloud cluster information under the preset viewing angle, that is, it can be understood as the point cloud cluster image being a vertical projection image of the point cloud cluster information on the ground.

[0126] S10322, Based on the point cloud cluster image and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0127] For example, please see Figure 8 , Figure 8 This is a schematic diagram of a point cloud cluster image and a preset roadblock shape provided in an embodiment of this disclosure. Figure 8 As shown in the figure, region 10 is the point cloud cluster image, and regions 20 and 30 can be point cloud cluster images of objects that obstruct the roadblock, such as trees or bushes. The rectangular frame formed by the dashed line is the preset roadblock shape. The two rectangular frames 100 shown in the figure can be understood as the preset roadblock shape at the starting position and the preset roadblock shape after rotating 40 degrees.

[0128] In some implementations, regarding step S10322 above, when determining the roadblock volume information and / or the roadblock orientation information based on the point cloud cluster image and the preset roadblock shape, such as... Figure 9 As shown, it may include the following S103221 to S103223:

[0129] S103221, Determine the center point of the point cloud cluster image, and set the center point of the point cloud cluster image as the center point of the preset roadblock shape.

[0130] For example, the center point can be determined based on the shape or area of ​​the point cloud cluster image, and the center point of the preset roadblock shape can be made to coincide with the center point of the point cloud cluster image.

[0131] S103222, rotate the preset roadblock shape around the center point multiple times, and determine the overlap area between the preset roadblock shape and the point cloud cluster image after each rotation.

[0132] For example, the preset roadblock shape can be rotated around the center point at the same angle each time, or it can be rotated at a different angle each time; there is no limitation on this.

[0133] For example, please see again Figure 8 The vertically upward direction can be used as the starting direction of the preset roadblock shape. The dashed rectangle 100 is rotated counterclockwise around the center point O, with each rotation angle being 20 degrees. After each rotation, the overlapping area between the preset roadblock shape at the current position and the point cloud cluster image must be determined. That is, the overlapping area between the dashed rectangle 100 at different positions and the irregular shape region 10 must be determined. Figure 8 The image shows the position after two rotations, that is, when the preset roadblock shape is rotated 40 degrees.

[0134] S103223, Based on the overlapping area between the preset roadblock shape after each rotation and the point cloud cluster image, determine the roadblock volume information and / or the roadblock orientation information.

[0135] It can be understood that when the overlap area between the preset roadblock shape and the point cloud cluster image is the largest, that is, when the orientation of the roadblock basically matches the orientation of the preset roadblock shape, the orientation information of the preset roadblock shape can be determined as the roadblock orientation information.

[0136] Secondly, since point cloud cluster information is 3D information, that is, it has spatial location information, when the overlap area between the preset roadblock shape and the point cloud cluster image is the largest, the volume of the roadblock corresponding to the point cloud cluster image is also the largest. Therefore, the volume of the point cloud cluster information corresponding to the point cloud cluster image can be determined based on the largest overlap area, and thus the roadblock volume information can be determined.

[0137] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0138] Based on the same inventive concept, this disclosure also provides a road obstacle detection device corresponding to the road obstacle detection method. Since the principle of the device in this disclosure for solving the problem is similar to the road obstacle detection method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0139] Please see Figure 10 This is a schematic diagram of the structure of a road obstacle detection device provided in an embodiment of the present disclosure. The device includes: a target detection module 710, a point cloud processing module 720, and a result determination module 730; wherein,

[0140] The target detection module 710 is used to perform target detection on the real-world scene image acquired by the image acquisition component to obtain the 2D detection result of the road obstacle;

[0141] The point cloud processing module 720 is used to obtain the point cloud cluster information of the roadblock based on the 2D detection results and the point cloud information corresponding to the real scene image;

[0142] The result determination module 730 is used to obtain the detection result of the road obstacle based on the point cloud cluster information and the preset road obstacle shape. The detection result of the road obstacle includes at least one of road obstacle location information, road obstacle volume information and road obstacle orientation information.

[0143] In one optional implementation, the point cloud processing module 720 is specifically used for:

[0144] Based on preset map information, the point cloud information is segmented to obtain target point cloud information;

[0145] Based on the 2D detection results and the target point cloud information, the point cloud cluster information of the roadblock is obtained.

[0146] In one optional implementation, the 2D detection result includes the display boundary of the obstacle in the real-world scene image; the point cloud processing module 720 is specifically used for:

[0147] Determine whether each point in the target point cloud information falls within the display boundary range of the roadblock;

[0148] Points in the target point cloud information that fall within the display boundary range of the roadblock are identified as the point cloud cluster information of the roadblock.

[0149] In one optional implementation, the point cloud processing module 720 is specifically used for:

[0150] Based on the absolute pose of the radar device collecting point cloud data, the relative pose of the image acquisition component with respect to the radar device, and the internal parameters of the image acquisition component, it is determined whether each point in the target point cloud information falls within the display boundary range of the roadblock.

[0151] In one optional implementation, the result determination module 730 is specifically used for:

[0152] The location information of the roadblock is determined based on the average depth information of the point cloud cluster information;

[0153] Based on the point cloud cluster information and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0154] In one optional implementation, the preset roadblock shape is the roadblock shape under a preset viewpoint; the result determination module 730 is specifically used for:

[0155] Determine the point cloud cluster information in the point cloud cluster image under the preset viewpoint;

[0156] Based on the point cloud cluster image and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

[0157] In one optional implementation, the result determination module 730 is specifically used for:

[0158] Determine the center point of the point cloud cluster image, and set the center point of the point cloud cluster image as the center point of the preset roadblock shape;

[0159] The preset roadblock shape is rotated multiple times around the center point, and the overlap area between the preset roadblock shape and the point cloud cluster image after each rotation is determined.

[0160] Based on the overlapping area between the preset roadblock shape after each rotation and the point cloud cluster image, the roadblock volume information and / or the roadblock orientation information are determined.

[0161] In one optional implementation, the result determination module 730 is specifically used for:

[0162] When the overlap area between the preset roadblock shape and the point cloud cluster image is maximized, the orientation information of the preset roadblock shape is determined as the roadblock orientation information; and / or,

[0163] When the area of ​​overlap between the preset roadblock shape and the point cloud cluster image is the largest, the volume of the point cloud cluster information corresponding to the largest overlap area is determined as the roadblock volume information.

[0164] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0165] Based on the same technical concept, embodiments of this disclosure also provide an electronic device. See also Figure 11 The diagram shown is a structural schematic of an electronic device 800 provided in an embodiment of this disclosure, including a processor 801, a memory 802, and a bus 803. The memory 802 is used to store execution instructions and includes a main memory 8021 and an external memory 8022. The main memory 8021, also called internal memory, is used to temporarily store computational data in the processor 801, as well as data exchanged with external memory 8022 such as a hard disk. The processor 801 exchanges data with the external memory 8022 through the main memory 8021.

[0166] In this embodiment, the memory 802 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 801. That is, when the electronic device 800 is running, the processor 801 communicates with the memory 802 through the bus 803, so that the processor 801 executes the application code stored in the memory 802, thereby executing the method in any of the foregoing embodiments.

[0167] Processor 801 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor.

[0168] The memory 802 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0169] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 800. In other embodiments of this application, the electronic device 800 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0170] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the obstacle detection method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0171] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the obstacle detection method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0172] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0173] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and terminals described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, terminals, and methods can be implemented in other ways. The terminal embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, or indirect coupling or communication connection between units, and may be electrical, mechanical, or other forms.

[0174] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0175] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0176] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0177] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for detecting road obstructions, characterized in that, include: Target detection is performed on the real-world scene images acquired by the image acquisition unit to obtain 2D detection results of road obstacles; Based on the 2D detection results and the point cloud information corresponding to the real scene image, the point cloud cluster information of the roadblock is obtained; Based on the point cloud cluster information and the preset roadblock shape, the detection result of the roadblock is obtained. The detection result of the roadblock includes at least one of roadblock location information, roadblock volume information, and roadblock orientation information. The preset roadblock shape is the roadblock shape under a preset viewpoint. The step of obtaining the roadblock detection result based on the point cloud cluster information and the preset roadblock shape includes: determining the point cloud cluster image of the point cloud cluster information under the preset viewpoint; and determining the roadblock volume information and / or the roadblock orientation information based on the point cloud cluster image and the preset roadblock shape.

2. The method according to claim 1, characterized in that, The step of obtaining the point cloud cluster information of the roadblock based on the 2D detection results and the point cloud information corresponding to the real-world scene image includes: Based on preset map information, the point cloud information is segmented to obtain target point cloud information; Based on the 2D detection results and the target point cloud information, the point cloud cluster information of the roadblock is obtained.

3. The method according to claim 2, characterized in that, The 2D detection result includes the display boundary of the roadblock in the real-world scene image; obtaining the point cloud cluster information of the roadblock based on the 2D detection result and the target point cloud information includes: Determine whether each point in the target point cloud information falls within the display boundary range of the roadblock; Points in the target point cloud information that fall within the display boundary range of the roadblock are identified as the point cloud cluster information of the roadblock.

4. The method according to claim 3, characterized in that, The step of determining whether each point in the target point cloud information falls within the display boundary range of the roadblock includes: Based on the absolute pose of the radar device collecting point cloud data, the relative pose of the image acquisition component with respect to the radar device, and the internal parameters of the image acquisition component, it is determined whether each point in the target point cloud information falls within the display boundary range of the roadblock.

5. The method according to claim 1, characterized in that, The step of obtaining the detection result of the road obstacle based on the point cloud cluster information and the preset road obstacle shape also includes: The location information of the roadblock is determined based on the average depth information of the point cloud cluster information.

6. The method according to claim 1, characterized in that, The step of determining the roadblock volume information and / or roadblock orientation information based on the point cloud cluster image and the preset roadblock shape includes: Determine the center point of the point cloud cluster image, and set the center point of the point cloud cluster image as the center point of the preset roadblock shape; The preset roadblock shape is rotated multiple times around the center point, and the overlap area between the preset roadblock shape and the point cloud cluster image after each rotation is determined. Based on the overlapping area between the preset roadblock shape after each rotation and the point cloud cluster image, the roadblock volume information and / or the roadblock orientation information are determined.

7. The method according to claim 6, characterized in that, The determination of the obstacle volume information and / or the obstacle orientation information based on the overlapping area between the preset obstacle shape after each rotation and the point cloud cluster image includes: When the overlap area between the preset roadblock shape and the point cloud cluster image is maximized, the orientation information of the preset roadblock shape is determined as the roadblock orientation information; and / or, When the area of ​​overlap between the preset roadblock shape and the point cloud cluster image is the largest, the volume of the point cloud cluster information corresponding to the largest overlap area is determined as the roadblock volume information.

8. A road obstacle detection device, characterized in that, include: The target detection module is used to perform target detection on the real-world scene images acquired by the image acquisition unit to obtain 2D detection results of road obstacles; The point cloud processing module is used to obtain the point cloud cluster information of the roadblock based on the 2D detection results and the point cloud information corresponding to the real scene image; The result determination module is used to obtain the detection result of the road obstacle based on the point cloud cluster information and the preset road obstacle shape. The detection result of the road obstacle includes at least one of road obstacle location information, road obstacle volume information, and road obstacle orientation information. The preset roadblock shape is the roadblock shape under a preset viewpoint, and the result determination module is also used to determine the point cloud cluster information as a point cloud cluster image under the preset viewpoint. Based on the point cloud cluster image and the preset roadblock shape, determine the roadblock volume information and / or the roadblock orientation information.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the obstacle detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the obstacle detection method as described in any one of claims 1 to 7.

Images