Obstacle detection method, device and equipment and storage medium

Abstract

The embodiment of the invention discloses an obstacle detection method and device, equipment and a storage medium. Comprising the steps of determining multi-modal features of a target based on point cloud data; wherein the multi-modal features comprise point cloud level features, target trajectory features and scene level features; logical reasoning is carried out on the multi-modal features to determine a first probability that the target is an obstacle; fusing the multi-modal features to determine a second probability that the target is an obstacle; and determining a final probability that the target is an obstacle based on the first probability and the second probability. According to the obstacle detection method provided by the embodiment of the invention, the final probability that the target is the obstacle is determined based on the first probability that the target is the obstacle and the second probability that the target is the obstacle, and the obstacle detection accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the field of driver assistance technology, and in particular to an obstacle detection method, device, equipment and storage medium. Background Technology

[0002] Millimeter-wave radar, due to its advantages such as accurate speed measurement and immunity to adverse weather conditions, has become a core sensor for advanced driver assistance systems (ADAS) and autonomous driving. However, existing millimeter-wave radar perception algorithms still have some shortcomings, making their target detection accuracy and precision insufficient for the demands of autonomous driving. Summary of the Invention

[0003] The present invention provides an obstacle detection method, apparatus, device, and storage medium, which can improve the accuracy of obstacle detection.

[0004] In a first aspect, embodiments of the present invention provide an obstacle detection method, comprising:

[0005] The multimodal features of the target are determined based on point cloud data; wherein, the multimodal features include point cloud-level features, target trajectory features, and scene-level features;

[0006] Logical reasoning is performed on the multimodal features to determine a first probability that the target is an obstacle;

[0007] The multimodal features are fused to determine a second probability that the target is an obstacle;

[0008] The final probability of the target being an obstacle is determined based on the first probability and the second probability.

[0009] Secondly, embodiments of the present invention also provide an obstacle detection device, comprising:

[0010] A multimodal feature determination module is used to determine the multimodal features of a target based on point cloud data; wherein, the multimodal features include point cloud-level features, target trajectory features, and scene-level features;

[0011] An obstacle first probability determination module is used to perform logical reasoning on the multimodal features to determine the first probability that the target is an obstacle;

[0012] The obstacle second probability determination module is used to fuse the multimodal features to determine the second probability that the target is an obstacle;

[0013] The obstacle final probability determination module is used to determine the final probability that the target is an obstacle based on the first probability and the second probability.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the obstacle detection method described in the embodiments of the present invention.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute the obstacle detection method described in the embodiments of the present invention.

[0019] This invention discloses an obstacle detection method, apparatus, device, and storage medium. The method determines multimodal features of a target based on point cloud data; these multimodal features include point cloud-level features, target trajectory features, and scene-level features. Logical reasoning is performed on the multimodal features to determine a first probability that the target is an obstacle; the multimodal features are fused to determine a second probability that the target is an obstacle; and a final probability that the target is an obstacle is determined based on the first and second probabilities. The obstacle detection method provided by this invention, which determines the final probability that the target is an obstacle based on the first and second probabilities, can improve the accuracy of obstacle detection. Attached Figure Description

[0020] Figure 1 This is a flowchart of an obstacle detection method according to Embodiment 1 of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of an obstacle detection device according to Embodiment 2 of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of an electronic device according to Embodiment 3 of the present invention. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0024] Example 1

[0025] Figure 1This is a flowchart of an obstacle detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where the probability of a target being an obstacle is determined. The method can be executed by an obstacle detection device, which can be implemented in software and / or hardware, optionally through an electronic device, such as a mobile terminal, PC, or server. Specifically, it includes the following steps:

[0026] S110, determining the multimodal features of the target based on point cloud data.

[0027] The multimodal features include point cloud-level features, target trajectory features, and scene-level features. Point cloud-level features include: point cloud pitch angle information, target size, target vertical distance information, point cloud radar cross section (RCS), signal-to-noise ratio (SNR), and number of clustered points; target trajectory features include: starting distance, target frame drop rate, target motion attributes, and SNR change trend; scene-level features include any one of the following: overpass scene, tunnel scene, scene where the moving target can pass through, or scene with curb or guardrail edge points.

[0028] The point cloud pitch angle information can be the standard deviation of the pitch angles of all points in the point cloud cluster, used to measure the height of the target. The calculation formula can be expressed as: ,in, Let be the pitch angle of the i-th point in the point cloud cluster. Let N be the average pitch angle of all points in the point cloud cluster, and let N represent the total number of points in the point cloud cluster. The larger the value, the more dispersed the point cloud is in the vertical direction, and the more extended the target is in terms of height. The smaller the value, the more compact the point cloud is in the vertical direction, and the more concentrated the target is in terms of height. Target dimensions include the length, width, and height of the target, determined based on the maximum and minimum values ​​of the point cloud cluster in the three dimensions of horizontal distance, vertical distance, and vertical distance. In this embodiment, the point cloud information is transformed from polar coordinates to Cartesian coordinates. The formulas for calculating the horizontal distance x, vertical distance y, and vertical distance z of the point cloud are as follows: , , Where Range is the radial distance of the point. Let be the azimuth angle of the dot. The pitch angle of the point trace. Target vertical distance information includes the average distance (z) of all points in the point cloud cluster along the z-axis. mean and the maximum value Z max Minimum value Z min If Z min –Z radar If the distance is greater than M meters (e.g., M=3.5), the target is likely a high-altitude, traversable target; if Z... max –Z radar<N meters (e.g., N = -0.3), then the target may be a ground-crossable target; Z radar is the radar installation height. The radar cross section (RCS) of the point cloud is represented by calculating the average value of the RCS of all traces in the point cloud cluster; in this application scenario, the RCS values of targets such as vehicles and signboards are relatively high, while the RCS values of ground-crossable targets (such as manhole covers and iron plates) are relatively low. The signal-to-noise ratio (SNR) can be represented by the average value or the maximum value of the SNR of all traces within the point cloud cluster; in this application scenario, the higher the SNR, the greater the probability that the target actually exists. The number of clustered traces is the number of traces included in the target; in this application scenario, the number of clustered traces is large and stable, which means that the target has strong reflection ability and clear contour (such as a vehicle); the number of clustered traces is small and flickering, which may be noise or a small target with a low RCS.

[0029] Among them, the starting distance is the radial distance between the target and the host vehicle when the target is first detected; in this application scenario, high-altitude fixed objects (such as overpasses) can be detected at a very far distance and maintain a relatively stable geometric shape. The starting distance of ground-crossable targets (such as manhole covers, iron plates, metal gaps, etc.) is very close (e.g., within 30m). The target dropout rate is the ratio of the number of frames in which the target is not detected by the radar to the total number of frames, which is used to distinguish fixed targets and flickering targets; if the dropout rate is extremely low, it is a fixed target that exists stably (such as a static vehicle or a signboard); if the dropout rate is relatively high, it may be noise or a small, short target (whose echo appears and disappears intermittently). The target motion attribute includes whether the target is a moving target or a stationary target; it can be determined based on the vehicle body information of the host vehicle (such as vehicle speed and vehicle yaw angle), as well as the point cloud radial velocity information and azimuth information. The SNR change trend includes an increase or a decrease in SNR. In this embodiment, the process of determining the SNR change trend can be as follows: 1. Calculate the average SNR of a set number of historical frames (e.g., 5 frames) as the SNR of the target in the current frame; 2. Calculate the maximum SNR of multiple historical frames; 3. Calculate the sum of the SNR differences between two consecutive frames of multiple historical frames; 4. Based on the sum of the SNR differences of multiple historical frames and the difference between the SNR of the target in the current frame and the maximum SNR, comprehensively evaluate the upward / downward trend of the SNR.

[0030] Among them, the overpass scenario has the following characteristics: a static point cloud cluster that spans the road and has a specific pitch angle distribution ( relatively high). Once an overpass is identified, the static point cloud directly below it can immediately be assigned a high probability of "traversable", that is, a low probability of being assigned as an obstacle. When entering a tunnel, the radar echo characteristics will change drastically (such as an increase in multiple reflections), and it has a specific pitch angle distribution ( (Higher performance); Inside tunnels, radar performance degrades, requiring adjustments to perception confidence or strategies, while filtering static targets such as tunnel walls. Moving target traversable scenarios are characterized by the trajectory of a static target that has previously been crossed or traversed by a dynamic target (vehicle). Curb or guardrail edge scenarios are characterized by targets that are consistently and precisely located at the curb / guardrail edge, such as guardrails, lampposts, and other roadside infrastructure, and cannot be crossed.

[0031] S120 performs logical reasoning on multimodal features to determine the first probability that the target is an obstacle.

[0032] Optionally, the method for performing logical reasoning on multimodal features to determine the first probability that the target is an obstacle can be: determining the target as a high-altitude target based on the target's vertical distance information; determining the high-altitude target as a non-obstacle when the high-altitude target satisfies any of the following: bridge scene, tunnel scene, or scene through which the moving target can pass; or, initializing the first probability that the high-altitude target is an obstacle; and adjusting the first probability based on at least one of the target's vertical distance information, SNR, SNR change trend, and target frame drop rate.

[0033] Specifically, when a target is determined to be a high-altitude target, if the high-altitude target satisfies any one of the following scenarios: a skybridge scenario, a tunnel scenario, or a scenario that a moving target can pass through, then the high-altitude target is determined to be a non-obstacle, i.e., the first probability that the target is an obstacle is 0. If the high-altitude target does not meet any one of the following scenarios: a skybridge scenario, a tunnel scenario, or a scenario that a moving target can pass through, then the first probability that the high-altitude target is an obstacle is initialized to 1. Specifically, the method of adjusting the first probability based on at least one of the following: target vertical distance information, SNR, SNR change trend, and target frame drop rate, can be: if the target vertical distance information satisfies a preset first height condition (e.g., z... mean If the target is a roadside or guardrail edge, and the SNR is higher than the first set value and the target's vertical distance information meets the preset second height condition, then the first probability is increased; if the SNR trend is downward, then the first probability is increased; if the SNR trend is upward, then the first probability is decreased; if the frame drop rate is greater than the third set value, then the first probability is decreased; if the frame drop rate is less than the fourth set value (e.g., 5%), then the first probability is increased.

[0034] Optionally, the method for performing logical reasoning on multimodal features to determine the first probability that the target is an obstacle can be: determining the target as a ground target based on the target's vertical distance information; determining the ground target as a non-obstacle when the ground target is a moving target that can pass through the scene; or, determining the ground target as a non-obstacle if the ground target meets the following conditions: the initial distance is less than a set distance, the SNR is greater than a set decibel, the RCS is less than a set threshold, and the target's vertical distance information meets a preset height condition; or initializing the first probability that the ground target is an obstacle; and adjusting the first probability based on the SNR and / or the target's frame drop rate.

[0035] When a target is determined to be a ground target, if the ground target is a moving target that can pass through the scene, or if the following conditions are met: the initial distance is less than a set distance, the SNR is greater than a set decibel, the RCS is less than a set threshold, and the target's vertical distance information meets a preset height condition, then the ground target is determined to be a non-obstacle, i.e., the first probability of the ground target being an obstacle is 0. If the ground target is not a moving target that can pass through the scene, and does not simultaneously meet the following conditions: the initial distance is less than a set distance, the SNR is greater than a set decibel, the RCS is less than a set threshold, and the target's vertical distance information meets a preset height condition, then the first probability of the ground target being an obstacle is initialized to 1. Specifically, the method for adjusting the first probability based on SNR and / or target frame drop rate can be as follows: if the target is a roadside or guardrail edge point, and the SNR is higher than the first set value and the target's vertical distance information meets the preset second height condition, then the first probability is increased; if the SNR trend is downward, then the first probability is increased; if the SNR trend is upward, then the first probability is decreased; if the frame drop rate is greater than the third set value, then the first probability is decreased; if the frame drop rate is less than the fourth set value (e.g., 5%), then the first probability is increased.

[0036] S130, fuses multimodal features to determine the second probability that the target is an obstacle.

[0037] Optionally, the method for fusing multimodal features to determine the second probability that the target is an obstacle can be: initializing the detection result type of the target; dividing the multimodal features into a first group of features and a second group of features; determining the first sub-probability of each detection result type based on the first group of features and a preset probability allocation function; determining the second sub-probability of each detection result type based on the second group of features and a preset probability allocation function; and determining the second probability that the target is an obstacle based on the first sub-probability and the second sub-probability.

[0038] The detection result types include: target is an obstacle, target is not an obstacle, target is either an obstacle or not an obstacle, and detection result is empty; these can be represented as A, B, C, and D, respectively. The first set of features includes: target SNR, target height, target RCS, and number of cluster points; the second set of features includes: target radial distance information, target lifespan, target frame drop rate, target starting distance, target velocity, and target size. The preset probability allocation function can be expressed as: , where C i For the weight, F i Let C be the i-th feature in either the first or second feature group. i The values ​​of the values ​​are different. Substituting the weights corresponding to the first set of features and the detection result types into the above probability allocation function, we obtain the first sub-probabilities of each detection result type, which can be expressed as: m1(A), m1(B), m1(C), and m1(D); substituting the weights corresponding to the second set of features and the detection result types into the above probability allocation function, we obtain the second sub-probabilities of each detection result type, which can be expressed as: m2(A), m2(B), m2(C), and m2(D).

[0039] Specifically, the method for determining the second probability of a target being an obstacle based on the first and second sub-probabilities can be as follows: First, determine the probability of the target being an obstacle, the probability of the target not being an obstacle, the probability of the target being an obstacle or not being an obstacle, and the probability of the detection result being empty based on the first and second sub-probabilities of each detection result type, respectively; finally, determine the reliability and similarity of the target being an obstacle based on the probability of the target being an obstacle, the probability of the target not being an obstacle, the probability of the target being an obstacle or not being an obstacle, and the probability of the detection result being empty, and determine the second probability of the target being an obstacle based on the reliability and similarity of the target being an obstacle.

[0040] The formula for calculating the probability that the target is an obstacle is as follows: The probability that the target is not an obstacle can be calculated using the following formula: The formula for calculating the probability that the target is an obstacle or a non-obstacle can be expressed as: The formula for calculating the probability of a test result being empty can be expressed as: .

[0041] In this embodiment, after obtaining m(A), m(B), m(C), and m(D), the final probabilities of A and C are determined according to the following formula: , Where K = m(A) + m(B) + m(C) + m(D). The reliability of A (the target is an obstacle) is... The similarity of A is Finally, the confidence level and similarity level of A are averaged to obtain the second probability that the target is an obstacle.

[0042] Optionally, after determining the second probability that the target is an obstacle based on the first sub-probability and the second sub-probability, the method further includes the following steps: obtaining the final probability that the target is an obstacle in the previous frame; and smoothing the second probability based on the final probability of the previous frame.

[0043] One method for smoothing the second probability based on the final probability of the previous frame is to perform a weighted summation of the final probability of the previous frame and the second probability of the target being an obstacle in the current frame to obtain the smoothed second probability.

[0044] S140, determine the final probability that the target is an obstacle based on the first probability and the second probability.

[0045] Optionally, the final probability of the target being an obstacle can be determined based on the first probability and the second probability by adjusting the first probability based on the second probability to obtain the final probability of the target being an obstacle.

[0046] Specifically, the way to adjust the first probability based on the second probability is: if the second probability is less than a set value, then the first probability is lowered.

[0047] In this embodiment, if the first probability and the second probability are both less than a set value (e.g., 0.1) in M ​​consecutive frames, the target is determined to be a non-obstacle, and the target can be passed through without further judgment.

[0048] In this embodiment, if the final probability that the target is an obstacle is greater than a set value (e.g., 0.8), the target is determined to be an obstacle so that the assisted driving system can make a decision.

[0049] This embodiment of the solution determines the multimodal features of a target based on point cloud data. These multimodal features include point cloud-level features, target trajectory features, and scene-level features. Logical reasoning is performed on the multimodal features to determine a first probability that the target is an obstacle. The multimodal features are then fused to determine a second probability that the target is an obstacle. Finally, the final probability that the target is an obstacle is determined based on the first and second probabilities. The obstacle detection method provided by this embodiment of the invention, which determines the final probability that the target is an obstacle based on the first and second probabilities, can improve the accuracy of obstacle detection.

[0050] Example 2

[0051] Figure 2 This is a schematic diagram of the structure of an obstacle detection device provided in Embodiment 2 of the present invention, as shown below. Figure 2 As shown, the device includes:

[0052] The multimodal feature determination module 210 is used to determine the multimodal features of a target based on point cloud data; wherein, the multimodal features include point cloud-level features, target trajectory features, and scene-level features;

[0053] The obstacle first probability determination module 220 is used to perform logical reasoning on the multimodal features to determine the first probability that the target is an obstacle;

[0054] The obstacle second probability determination module 230 is used to fuse the multimodal features to determine the second probability that the target is an obstacle;

[0055] The obstacle final probability determination module 240 is used to determine the final probability that the target is an obstacle based on the first probability and the second probability.

[0056] Optionally, the point cloud-level features include: point cloud pitch angle information, target size, target vertical distance information, point cloud radar cross section (RCS), signal-to-noise ratio (SNR), and number of clustered points; the target trajectory features include: starting distance, target frame drop rate, target motion attributes, and SNR change trend; the scene-level features include any one of the following: overpass scene, tunnel scene, scene where the moving target can pass through, and scene of curb or guardrail edge points.

[0057] Optionally, the obstacle first probability determination module 220 is also used for:

[0058] Based on the target's vertical distance information, the target is determined to be a high-altitude target;

[0059] The high-altitude target is determined to be a non-obstacle if it meets any of the following conditions: it is located in a pedestrian bridge scene, a tunnel scene, or a scene through which a moving target can pass; or...

[0060] Initialize the first probability that the high-altitude target is an obstacle; and adjust the first probability based on at least one of the target's vertical distance information, SNR, SNR change trend, and target frame drop rate.

[0061] Optionally, the obstacle first probability determination module 220 is also used for:

[0062] Based on the target's vertical distance information, the target is determined to be a ground target;

[0063] If the ground target is a moving target that can pass through the scene, then the ground target is determined to be a non-obstacle; or,

[0064] If the ground target meets the following conditions: the initial distance is less than a set distance, the SNR is greater than a set decibel, the RCS is less than a set threshold, and the target's vertical distance information meets a preset height condition, then the ground target is determined to be a non-obstacle; or

[0065] Initialize the ground target as an obstacle with a first probability; and adjust the first probability based on SNR and / or target frame drop rate.

[0066] Optionally, the obstacle second probability determination module 230 is also used for:

[0067] Initialize the target detection result type; wherein, the detection result type includes: target is an obstacle, target is not an obstacle, target is an obstacle or not an obstacle, and detection result is empty;

[0068] The multimodal features are divided into a first group of features and a second group of features;

[0069] The first sub-probability of each detection result type is determined based on the first set of features and the preset probability allocation function; the second sub-probability of each detection result type is determined based on the second set of features and the preset probability allocation function.

[0070] A second probability is determined based on the first sub-probability and the second sub-probability to identify the target as an obstacle.

[0071] Optionally, it also includes: a smoothing module for:

[0072] Obtain the final probability that the target is an obstacle in the previous frame;

[0073] The second probability is smoothed based on the final probability of the previous frame.

[0074] Optionally, the obstacle final probability determination module 240 is also used for:

[0075] The first probability is adjusted based on the second probability to obtain the final probability that the target is an obstacle.

[0076] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in this embodiment can be found in the methods provided in all the foregoing embodiments of the present invention.

[0077] Example 3

[0078] Figure 3A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components, connections, relationships, and functions shown herein are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0079] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0080] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0081] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as obstacle detection methods.

[0082] In some embodiments, the obstacle detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the obstacle detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the obstacle detection method by any other suitable means (e.g., by means of firmware).

[0083] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0084] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0085] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0086] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0087] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0088] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0089] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the obstacle detection method provided in any embodiment of this application.

[0090] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0091] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0092] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An obstacle detection method, characterized in that, include: The multimodal features of the target are determined based on point cloud data; wherein, the multimodal features include point cloud-level features, target trajectory features, and scene-level features; Logical reasoning is performed on the multimodal features to determine a first probability that the target is an obstacle; The multimodal features are fused to determine a second probability that the target is an obstacle; The final probability of the target being an obstacle is determined based on the first probability and the second probability.

2. The method according to claim 1, characterized in that, The point cloud-level features include: point cloud pitch angle information, target size, target vertical distance information, point cloud radar cross section (RCS), signal-to-noise ratio (SNR), and number of clustered points; the target trajectory features include: starting distance, target frame drop rate, target motion attributes, and SNR change trend; the scene-level features include any one of the following: overpass scene, tunnel scene, scene through which moving targets can pass, and scene at the edge of a curb or guardrail.

3. The method according to claim 2, characterized in that, Performing logical reasoning on the multimodal features to determine a first probability that the target is an obstacle includes: Based on the target's vertical distance information, the target is determined to be a high-altitude target; The high-altitude target is determined to be a non-obstacle if it meets any of the following conditions: it is located in a pedestrian bridge scene, a tunnel scene, or a scene through which a moving target can pass; or... Initialize the first probability that the high-altitude target is an obstacle; and adjust the first probability based on at least one of the target's vertical distance information, SNR, SNR change trend, and target frame drop rate.

4. The method according to claim 2, characterized in that, Performing logical reasoning on the multimodal features to determine a first probability that the target is an obstacle includes: Based on the target's vertical distance information, the target is determined to be a ground target; If the ground target is a moving target that can pass through the scene, then the ground target is determined to be a non-obstacle; or, If the ground target meets the following conditions: the initial distance is less than a set distance, the SNR is greater than a set decibel, the RCS is less than a set threshold, and the target's vertical distance information meets a preset height condition, then the ground target is determined to be a non-obstacle; or Initialize the ground target as an obstacle with a first probability; and adjust the first probability based on SNR and / or target frame drop rate.

5. The method according to claim 1, characterized in that, The multimodal features are fused to determine a second probability that the target is an obstacle, including: Initialize the target detection result type; wherein, the detection result type includes: target is an obstacle, target is not an obstacle, target is an obstacle or not an obstacle, and detection result is empty; The multimodal features are divided into a first group of features and a second group of features; The first sub-probability of each detection result type is determined based on the first set of features and the preset probability allocation function; the second sub-probability of each detection result type is determined based on the second set of features and the preset probability allocation function. A second probability is determined based on the first sub-probability and the second sub-probability to identify the target as an obstacle.

6. The method according to claim 5, characterized in that, After determining a second probability that the target is an obstacle based on the first sub-probability and the second sub-probability, the method further includes: Obtain the final probability that the target is an obstacle in the previous frame; The second probability is smoothed based on the final probability of the previous frame.

7. The method according to claim 1, characterized in that, Determining the final probability that the target is an obstacle based on the first probability and the second probability includes: The first probability is adjusted based on the second probability to obtain the final probability that the target is an obstacle.

8. An obstacle detection device, characterized in that, include: A multimodal feature determination module is used to determine the multimodal features of a target based on point cloud data; wherein, the multimodal features include point cloud-level features, target trajectory features, and scene-level features; An obstacle first probability determination module is used to perform logical reasoning on the multimodal features to determine the first probability that the target is an obstacle; The obstacle second probability determination module is used to fuse the multimodal features to determine the second probability that the target is an obstacle; The obstacle final probability determination module is used to determine the final probability that the target is an obstacle based on the first probability and the second probability.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the obstacle detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the obstacle detection method according to any one of claims 1-7.

Images