Dynamic and static obstacle detection method, system, equipment and medium

Abstract

The embodiment of the invention relates to the technical field of intelligent driving, and discloses a dynamic and static obstacle detection method, system and device and a medium. Performing message synchronization on the 4D millimeter wave radar point cloud data, the vehicle speed and the course information based on the timestamp; performing point cloud correction on the 4D millimeter wave radar point cloud data, and segmenting non-ground point cloud data; performing single-frame obstacle detection on the non-ground point cloud data to obtain a current-frame detection obstacle; and according to the vehicle speed and course information, a track management strategy is adopted to associate and match the current frame detection obstacle with the previous frame track obstacle, the track is correspondingly processed and updated based on the association result, the track attribute is judged, and the track obstacle is output. The technical problem of inaccurate static obstacle detection can be at least solved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a method, system, device and medium for detecting dynamic and static obstacles. Background Technology

[0002] In the field of intelligent driving technology, obstacle detection is a crucial component in ensuring the safety and decision-making capabilities of autonomous driving systems. Autonomous vehicles need to autonomously identify various obstacles on the road in complex traffic environments to make correct path planning and decisions. Obstacle detection not only requires recognizing the presence of obstacles but also accurately locating their position, size, type, and dynamic or static characteristics. This provides the decision-making module with more detailed environmental information, ensuring driving safety and smoothness.

[0003] Current obstacle detection solutions mainly include two mainstream approaches: deep learning-based network models and rule-based traditional algorithms. Deep learning methods typically use large labeled datasets to train neural network models, which then detect and classify obstacles. This approach has good adaptability and can handle complex environmental changes, but it also has some drawbacks. For example, deep learning models require a large amount of training data, and building these datasets is often very costly. Furthermore, existing deep learning models can only identify obstacle categories included in the training dataset, thus failing to handle obstacle categories not covered in the training dataset, leading to significant missed detections. Especially for static obstacles such as traffic cones, bollards, guardrails, and water-filled barriers, existing deep learning methods are often ineffective, even though these obstacles are crucial for traffic safety in real-world road environments.

[0004] On the other hand, traditional obstacle detection methods based on clustering and occupancy grids identify and locate obstacles by clustering point cloud data. The advantages of this method are that it does not require a large amount of training data, has low cost, and can be implemented quickly. However, due to its relatively simplified approach, traditional clustering methods are prone to false detections when dealing with dynamic obstacles in complex environments, especially for the classification of some dynamic obstacles, which can easily lead to incorrect identification results and thus affect vehicle decision-making and path planning. Summary of the Invention

[0005] One objective of this application is to provide a method, system, device, and medium for detecting dynamic and static obstacles, at least to solve the technical problem of inaccurate detection of dynamic and static obstacles.

[0006] To achieve the above objectives, some embodiments of this application provide the following aspects:

[0007] Firstly, some embodiments of this application also provide a method for detecting dynamic and static obstacles, including acquiring 4D millimeter-wave radar point cloud data, vehicle speed, and heading information; synchronizing the 4D millimeter-wave radar point cloud data, vehicle speed, and heading information based on timestamps; performing point cloud correction on the 4D millimeter-wave radar point cloud data to segment out non-ground point cloud data; performing single-frame obstacle detection on the non-ground point cloud data, including obstacle detection of objects of interest and obstacles of non-interest; matching the detection results to distinguish between obstacles of interest and obstacles of non-interest, obtaining the obstacles detected in the current frame; and, based on the vehicle speed and heading information, using a trajectory management strategy to associate and match the obstacles detected in the current frame with the obstacles in the trajectory of the previous frame, updating the trajectory based on the association results, judging the trajectory attributes, and outputting the obstacles in the trajectory.

[0008] Secondly, some embodiments of this application also provide a dynamic and static obstacle detection system, including a message synchronization module for acquiring 4D millimeter-wave radar point cloud data, vehicle speed, and heading information; synchronizing the 4D millimeter-wave radar point cloud data, vehicle speed, and heading information based on timestamps; a point cloud preprocessing module for performing point cloud correction on the 4D millimeter-wave radar point cloud data and segmenting non-ground point cloud data; a single-frame obstacle detection module for performing single-frame obstacle detection on the non-ground point cloud data, including obstacle detection of objects of interest and obstacles of non-interest; matching the detection results to distinguish between obstacles of interest and obstacles of non-interest, and obtaining the obstacles detected in the current frame; and an obstacle tracking and attribute judgment module for associating and matching the obstacles detected in the current frame with the obstacles in the previous frame using a trajectory management strategy based on the vehicle speed and heading information, updating the trajectory accordingly based on the association results, judging the trajectory attributes, and outputting the obstacles in the trajectory.

[0009] Thirdly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0010] Fourthly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods described above.

[0011] Compared with related technologies, the solution provided in this application, by fusing 4D millimeter-wave radar point cloud data, vehicle speed, and heading information from multiple sources, and combining deep learning models and traditional clustering algorithms, can more accurately identify and distinguish dynamic and static obstacles in dynamic environments, thereby effectively reducing the probability of missed detections and false detections. It not only provides the location, size, and category of obstacles, but also their dynamic and static attributes, offering more comprehensive information support for the intelligent driving system and helping the system make safer and smoother decisions. Attached Figure Description

[0012] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0013] Figure 1 This is a flowchart illustrating a method for detecting dynamic and static obstacles according to an embodiment of this application;

[0014] Figure 2 This is a flowchart illustrating another method for detecting dynamic and static obstacles according to an embodiment of this application;

[0015] Figure 3 This is a schematic diagram of the structure of a dynamic and static obstacle detection system according to an embodiment of this application;

[0016] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] First Embodiment

[0019] The first embodiment of this application relates to a method for detecting dynamic and static obstacles. For example... Figure 1 As shown, the method may include the following steps:

[0020] S101, acquire 4D millimeter-wave radar point cloud data, vehicle speed and heading information; synchronize the 4D millimeter-wave radar point cloud data, vehicle speed and heading information based on timestamps.

[0021] By synchronizing 4D millimeter-wave radar point cloud data with vehicle speed and heading information, the system ensures that the acquired obstacle data matches the vehicle's dynamic state. This precise synchronization provides time consistency for subsequent obstacle detection, avoiding errors that may arise from different sensor data timings, thereby improving the system's accuracy and robustness. Especially in high-speed driving or complex environments, precise synchronization can effectively reduce missed detections and misjudgments of dynamic obstacles.

[0022] S102, perform point cloud correction on the 4D millimeter-wave radar point cloud data and segment out the non-ground point cloud data.

[0023] Point cloud correction and non-ground point cloud segmentation ensure the accuracy of point cloud data in the vehicle coordinate system, avoiding calculation errors caused by angular deviations or sensor positioning errors in 4D millimeter-wave radar data. By removing ground point clouds, the system can focus on obstacle-related data, reducing computational load and improving obstacle detection efficiency. Furthermore, non-ground point cloud segmentation effectively identifies obstacle locations, improving detection accuracy.

[0024] S103, perform single-frame obstacle detection on the non-ground point cloud data, including obstacle detection of interest and obstacle detection of non-interest; match the detection results to distinguish between obstacles of interest and obstacles of non-interest, and obtain the obstacles detected in the current frame.

[0025] By combining the classification and detection of obstacles of interest (such as dynamic obstacles like cars and pedestrians) with those of non-interest (such as construction obstacles and static facilities), not only is the detection accuracy improved, but false positives are also reduced. For dynamic obstacles, deep learning methods can effectively identify their category and location, providing more accurate information for the decision-making system; for static obstacles, traditional clustering methods can effectively distinguish these stationary obstacles, thus avoiding decision-making errors caused by misclassification.

[0026] S104, based on the vehicle speed and heading information, a trajectory management strategy is used to associate and match the obstacle detected in the current frame with the obstacle in the trajectory of the previous frame, and the trajectory is updated accordingly based on the association result. The trajectory attributes are judged and the obstacle in the trajectory is output.

[0027] The trajectory management strategy effectively eliminates errors and noise in single-frame obstacle detection, especially in complex scenarios involving high-speed travel or multiple obstacles, preventing false detection and loss of obstacles. Through the correlation and matching of multi-frame data, the system can track the movement trajectory of obstacles, thereby improving the stability and accuracy of obstacle detection in dynamic scenes. Furthermore, by judging the dynamic and static attributes of obstacles in the trajectory, the system can distinguish between stationary and dynamic obstacles, providing richer contextual information for the intelligent decision-making system and effectively supporting path planning and decision-making.

[0028] It is readily apparent that, compared to related technologies, the solution provided in this application combines multi-source information (4D millimeter-wave radar, vehicle speed, heading, etc.) with the complementary advantages of deep learning and traditional clustering methods, ensuring high accuracy and robustness in obstacle detection. Synchronized data, accurate correction, effective classification, and trajectory management enable it to adapt to dynamic environments, reducing missed and false detections. Especially under complex road conditions, it provides stable and reliable obstacle detection capabilities, significantly improving the safety and decision-making accuracy of intelligent driving systems.

[0029] Second Embodiment

[0030] The second embodiment of this application relates to a method for detecting dynamic and static obstacles. The second embodiment is an improvement upon the first embodiment, such as… Figure 2 As shown, the specific improvements are as follows:

[0031] Detection is based on multi-source information, including point cloud data from a 4D millimeter-wave radar sensor, vehicle speed, and heading information (from onboard bus information or combined inertial navigation sensors). Three types of data are continuously received: the 4D millimeter-wave radar point cloud is the master metadata; upon receiving point cloud data, a synchronization mechanism is triggered, outputting the point cloud message closest to the point cloud frame timestamp, along with vehicle speed and heading information; if no point cloud data is received, the synchronization mechanism is not executed, and no information is output. The synchronized output information is then used by downstream modules.

[0032] Furthermore, point cloud correction includes performing angle correction on the 4D millimeter-wave radar point cloud data based on the angle correction file, converting it into a structured point cloud; and transferring the 4D millimeter-wave radar point cloud data to the vehicle coordinate system based on the 4D millimeter-wave radar extrinsic parameters.

[0033] Due to the disordered arrangement of the original point cloud data of 4D millimeter-wave radar, the angular offset of the point cloud, and the coordinates being in the 4D millimeter-wave radar's own coordinate system, the point cloud needs to be corrected before subsequent processing. This includes angular correction based on the angle correction file and converting it into a structured point cloud, and then transferring the point cloud to the vehicle's coordinate system based on the 4D millimeter-wave radar's extrinsic parameters to facilitate subsequent processing.

[0034] Furthermore, segmenting the non-ground point cloud data includes: processing the 4D millimeter-wave radar point cloud data through platform fitting, grid occupancy, clustering, and region growing to segment ground points and non-ground points, and outputting the non-ground point cloud data.

[0035] Since obstacles are non-ground point clouds, point cloud segmentation is necessary to reduce the amount of point cloud processing and avoid detection errors caused by ground points, thus removing ground point clouds. Ground and non-ground point cloud segmentation can be performed using methods such as plane fitting, grid occupancy, clustering, and region growing, and the non-ground point cloud data can then be output to downstream modules.

[0036] Furthermore, the obstacle of interest detection includes: detecting obstacles of interest in the non-ground point cloud data using an obstacle of interest network model; establishing a training dataset by collecting and labeling the data of the obstacles of interest; training and evaluating the obstacle of interest network model based on the training dataset; deploying the trained obstacle of interest network model in the vehicle-side computing platform, performing inference based on the non-ground point cloud data, and outputting the position, heading angle, size, and category of the obstacle of interest in the vehicle coordinate system.

[0037] Common, movable, easy-to-collect, and easy-to-label obstacles on roads are generally referred to as obstacles of interest, which mainly include five categories: cars (all types of passenger vehicles), large vehicles (vans, trucks, buses), tricycles, two-wheeled riders (motorcycles, electric vehicles, bicycles), and pedestrians.

[0038] Data on obstacles of interest are collected and labeled to create a training dataset. A supervised network model is then trained and evaluated. The trained model is deployed on an on-vehicle computing platform, performing inference based on non-terrestrial point clouds to output the obstacle's position (XYZ), heading angle, dimensions (length, width, height), and category attributes in the vehicle coordinate system. The supervised network model can be trained and deployed using models such as PointPillar and PointNet.

[0039] Due to limitations in the training dataset, the network model detection supports outputting obstacles of interest. These obstacles are mostly movable (dynamic or static). Because they are movable, the detection requirements for these obstacles are high, and the decision module is used to predict the movement trend of these obstacles and make reasonable plans.

[0040] Furthermore, the non-interest obstacle detection includes: clustering the non-ground point cloud data, extracting and transforming features for each point space or local space, and segmenting the point cloud of the non-interest obstacle with different feature attributes; the feature attributes include normal vector, density, distance, and reflection intensity; the clustering includes Euclidean clustering and density clustering.

[0041] The network model generally cannot detect stationary obstacles, such as traffic cones, stone blocks, piles of stones and sand, construction facilities, guardrails, water-filled barriers, etc., which require traditional clustering methods for detection.

[0042] For 3D point cloud clustering, clustering is typically performed using its characteristic attributes. Features are extracted or transformed for each point in space or a local space to obtain multiple attributes, such as normal vector, density, distance, and reflection intensity, thus segmenting the point cloud with different attributes. Commonly used clustering methods in point cloud processing include Euclidean clustering and density clustering.

[0043] Taking Euclidean clustering as an example, based on the Euclidean example as the criterion, for a point P in space, a KD-Tree is used to search for k nearest points to P in the neighborhood. Points with a distance less than a set threshold are added to set Q. If no more elements are added to Q, the clustering ends; otherwise, points other than P are selected from set Q, and the above steps are repeated. 4D millimeter-wave radar exhibits a near-dense, far-sparse characteristic during sampling, resulting in smaller distances between nearby point sets and larger distances between distant point sets. To avoid good clustering results for nearby obstacles but under-segmentation or truncation of distant obstacles, a segmented clustering method can be used. Different distance thresholds are set for different clustering segments, and the clustering algorithm for each segment is run in parallel using multiple threads. Finally, the clustered obstacles are fused at the segment intervals.

[0044] To avoid abnormal point cloud clusters in the clustering results, it is necessary to implement threshold management of the clustering results based on experience, set thresholds for size and location, and remove clustering results that are too large or floating in the air.

[0045] Obstacle detection and matching involves obtaining two obstacle detection results for the same frame of non-ground point cloud data: a network model and traditional clustering. These two detection results are then matched, and obstacles that match the network model detection results are removed from the clustering results. All network model results are then considered obstacles of interest, while the remaining clustering results are considered obstacles of non-interest, each carrying an interest / non-interest attribute and passed to downstream modules.

[0046] Multi-obstacle association matching algorithms include IOU-based matching, nearest neighbor algorithms, probabilistic data association, Hungarian matching algorithms, and KM algorithms, each with its own advantages and disadvantages. The industry commonly uses the KM algorithm, also known as weighted Hungarian matching. The weight is a distance metric, which can be calculated using the IOU value of the top-view projections of the two obstacle 3D bounding boxes, the Euclidean distance between their center points, etc. The association threshold can be adjusted based on algorithm debugging.

[0047] To avoid inaccurate or jittery results in a single frame, multi-frame tracking is required to ensure the accuracy of obstacle detection and the stability of obstacle attributes, thus improving the algorithm's robustness. Obstacles detected in the upstream single frame are used as obstacles detected in the current frame. A trajectory management strategy is employed to associate and match obstacles detected in the current frame with obstacles in the previous frame's trajectory. Based on the association results, the trajectory is updated accordingly, and trajectory attributes are evaluated before the final trajectory result is output.

[0048] The detection of obstacles in the flight path involves inter-frame correlation. Since the previous frame is earlier than the current frame, the Kalman filter algorithm is used to predict the position and heading of the obstacle in the current frame based on the position, heading, and speed of the obstacle in the previous frame in the vehicle's coordinate system, and then the obstacle attribute information is updated.

[0049] Since there are no historical tracks in the first frame, prediction is not required, and all detected obstacles in the first frame are used as the initial tracks. Because most obstacles of non-interest are stationary and have no absolute speed or heading, only relative motion with respect to the vehicle, the vehicle's speed and heading information are reversed and used as the speed and heading of the obstacles of non-interest.

[0050] Both obstacle detection and path obstacle detection include two types: those of interest and those of non-interest. During the association matching process, cascaded association matching is performed: first, obstacles of interest are matched with obstacles of interest in the path; second, obstacles of non-interest are matched with obstacles of non-interest in the path; finally, the remaining unmatched obstacles and path obstacles are matched. This ensures that obstacles of the same type are matched first, improving matching accuracy and reducing the amount of computation involved.

[0051] The association matching algorithm still uses the weighted Hungarian matching algorithm, and the distance metric can still be measured using the IOU value of the top-view projection of the two obstacle 3D bounding boxes into 2D bounding boxes, the Euclidean distance between the center points, etc. The association threshold setting can be adjusted according to the algorithm debugging situation.

[0052] Furthermore, the trajectory management strategy includes: handling associated trajectory obstacles, handling unassociated trajectory obstacles, and handling unassociated detected obstacles.

[0053] Furthermore, the method includes: the track status includes a determined track, a pending track, and an invalid track; the determined track is the final output track; the invalid track is the track to be deleted; the pending track becomes a determined track as the number of associated frames increases, and becomes an invalid track as the number of associated frames decreases.

[0054] The detected obstacles and track obstacles matched with the association are updated in trajectory, and the association frame number is incremented by one; when the association frame number meets the preset track determination threshold (e.g., 5 frames), the track status is updated to a determined track; the obstacle category is updated based on the DS evidence theory algorithm; the motion information is updated based on the Kalman filter algorithm; the shape and size information is updated through logical judgment (e.g., averaging).

[0055] When an unassociated track is a pending track, the number of associated frames is decremented by one; when the number of consecutive unassociated frames meets the preset invalid track threshold (e.g., 3 frames), the track status is updated to invalid track; the category and size information of obstacles in the unassociated track remain unchanged, and the motion information is updated through Kalman filtering.

[0056] If no obstacle is detected, a new track is initialized. The track status is "pending track" and it is added to the track pool.

[0057] The track attribute determination function inputs the vehicle's absolute speed and heading. Knowing the relative speed of the obstacle relative to the vehicle, the function can determine the obstacle's absolute speed. An absolute speed of 0 indicates a static obstacle; otherwise, it is considered a dynamic obstacle. This avoids the problem of classifying obstacles that are relatively stationary relative to the vehicle as static obstacles when the vehicle is moving.

[0058] The track output outputs the obstacles in the "track determined" state. The obstacles have rich information such as category (each category of interest, general obstacles), size (length, width and height), motion (position, speed, heading angle), dynamic and static status, which facilitates the downstream decision module to make a safe and smooth planning strategy.

[0059] It is not difficult to see that, in the embodiments of this application, by using multi-source information such as 4D millimeter-wave radar point cloud, vehicle speed and heading, and through a combination of network model and traditional clustering and trajectory management, the false negative and false positive rates of dynamic and static obstacles are reduced, the model training cost is reduced, the amount of point cloud data processing is reduced, and the accuracy and robustness of obstacle detection are improved; the dynamic and static attributes of obstacles are increased, providing more effective information for the decision-making module.

[0060] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0061] Third Embodiment

[0062] The third embodiment of this application relates to a dynamic and static obstacle detection system / device, such as... Figure 3As shown, the system includes:

[0063] The message synchronization module is used to acquire 4D millimeter-wave radar point cloud data, vehicle speed, and heading information; and to synchronize the 4D millimeter-wave radar point cloud data, vehicle speed, and heading information based on timestamps.

[0064] The point cloud preprocessing module is used to perform point cloud correction on the 4D millimeter-wave radar point cloud data and segment out non-ground point cloud data.

[0065] The single-frame obstacle detection module is used to perform single-frame obstacle detection on the non-ground point cloud data, including obstacle detection of interest and obstacle detection of non-interest; the detection results are matched to distinguish between obstacles of interest and obstacles of non-interest to obtain the obstacles detected in the current frame;

[0066] The obstacle tracking and attribute judgment module is used to associate and match the obstacles detected in the current frame with the obstacles in the previous frame using a trajectory management strategy based on the vehicle speed and heading information, update the trajectory accordingly based on the association results, judge the trajectory attributes, and output the obstacles in the trajectory.

[0067] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0068] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problem proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0069] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0070] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 4 An exemplary structural diagram of the electronic device is disclosed. For example... Figure 4 As shown, the electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0071] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103, and output device 1104 may be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0072] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0073] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0074] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0075] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0076] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0077] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0078] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0079] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0080] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, or similar devices. Furthermore, some steps or functions of this application can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.

[0081] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0082] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0083] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method of detecting dynamic and static obstacles, characterized in that, The method comprises: acquiring 4D millimeter wave radar point cloud data, vehicle speed and heading information; synchronizing the 4D millimeter wave radar point cloud data, vehicle speed and heading information based on timestamps; performing point cloud correction on the 4D millimeter wave radar point cloud data, and segmenting non-ground point cloud data; performing single-frame obstacle detection on the non-ground point cloud data, including interested obstacle detection and non-interested obstacle detection; matching the detection results to distinguish interested obstacles and non-interested obstacles, and obtaining current frame detection obstacles; based on the vehicle speed and heading information, using a track management strategy to associate and match the current frame detection obstacles with the last frame track obstacles, updating the track based on the association result, judging the track attributes, and outputting the track obstacles.

2. The method of claim 1, wherein, The point cloud correction comprises: performing angle correction on the 4D millimeter wave radar point cloud data based on an angle correction file, and converting it into structured point cloud; converting the 4D millimeter wave radar point cloud data into a vehicle coordinate system based on 4D millimeter wave radar external parameters.

3. The method of claim 2, wherein, The segmentation of non-ground point cloud data comprises: processing the 4D millimeter wave radar point cloud data through platform fitting, occupancy grid, clustering and region growing, segmenting ground points and non-ground points, and outputting the non-ground point cloud data.

4. The method of claim 3, wherein, The interested obstacle detection comprises: detecting interested obstacles in the non-ground point cloud data through an interested obstacle network model; collecting and labeling data of interested obstacles to establish a training data set; training and evaluating the interested obstacle network model based on the training data set; deploying the trained interested obstacle network model in a vehicle-side computing platform, performing inference based on the non-ground point cloud data, and outputting the position, heading angle, size and category of the interested obstacles in the vehicle coordinate system.

5. The method of claim 4, wherein, The non-interested obstacle detection comprises: clustering the non-ground point cloud data, extracting and converting features of each point space or local space, and segmenting point clouds of non-interested obstacles with different feature attributes. The feature attributes include normal vector, density, distance and reflection intensity; the clustering includes Euclidean clustering and density clustering.

6. The method of claim 5, wherein, The track management strategy comprises: associated track obstacle processing, unassociated track obstacle processing and unassociated detection obstacle processing.

7. The method of claim 6, wherein, The method comprises: The track state comprises determined track, pending track and invalid track; the determined track is the final output track; the invalid track is the track to be deleted; the pending track becomes a determined track as the number of associated frames increases, and becomes an invalid track as the number of associated frames decreases; updating the trajectory of the detection obstacles and track obstacles matched by association, and increasing the number of associated frames by one; when the number of associated frames meets the preset determined track threshold, updating the track state to a determined track; when the unassociated track is a pending track, the number of associated frames is reduced by one; when the number of consecutive unassociated frames meets the preset invalid track threshold, the track state is updated to an invalid track; unassociated detection obstacles initialize a new track, the track state is a pending track, and are added to the track pool.

8. A dynamic-static obstacle detection system characterized by, The system comprises: a message synchronization module, configured to acquire 4D millimeter wave radar point cloud data, vehicle speed and heading information; and perform message synchronization on the 4D millimeter wave radar point cloud data, vehicle speed and heading information based on timestamps; a point cloud preprocessing module, configured to perform point cloud correction on the 4D millimeter wave radar point cloud data, and segment out non-ground point cloud data; a single-frame obstacle detection module, configured to perform single-frame obstacle detection on the non-ground point cloud data, including interested obstacle detection and non-interested obstacle detection; and perform matching on detection results to distinguish interested obstacles and non-interested obstacles, and obtain current frame detection obstacles; an obstacle tracking and attribute judgment module, configured to associate and match the current frame detection obstacles with last frame track obstacles according to the vehicle speed and heading information by using a track management strategy, update tracks based on association results, judge track attributes, and output track obstacles.

9. An electronic device, comprising: The electronic device comprises: one or more processors; and a memory storing computer program instructions which, when executed, cause the processor to perform the steps of the method of any one of claims 1 to 7.

10. A computer readable medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method of any one of claims 1 to 7.

Images