Free space real-time detection method based on quasi-density tree structure

Abstract

The present invention provides a real-time free space detection method based on a quasi-density tree structure, comprising: extracting point cloud data at a specific spatial height based on the vehicle's height, and gridding the point cloud data; establishing a distance dictionary and an angle dictionary for each grid node relative to the center of the coordinate system, and establishing a subordinate point cloud dictionary for each grid node; establishing a dynamic matrix, a static matrix, a node density matrix, and a node occupancy matrix to evaluate the probability of target presence at each node; obtaining dynamic and static target presence probability matrices and determining a comprehensive score matrix; using the distance dictionary and angle dictionary, comparing the points closest to the coordinate center in the subordinate point clouds of the nearest grid node, and drawing and outputting a drivable area image based on the coordinates of the points. The real-time free space detection method, device, and computer-readable storage medium based on a quasi-density tree structure provided by the present invention simultaneously identify targets in an environment and free space.

Description

Technical Field

[0001] The present invention relates to the field of machine perception technology in autonomous driving, and in particular to a free space real-time detection method, device, and computer-readable storage medium based on a quasi-density tree structure. Background Art

[0002] The description of the background technology in the present invention belongs to the related technology related to the present invention and is only used to illustrate and facilitate the understanding of the invention content of the present invention. It should not be understood that the applicant explicitly believes or infers that the applicant believes that it is the prior art of the present invention on the filing date of the first application.

[0003] Environmental perception is a key machine perception task in the autonomous driving field, and identifying drivable areas in the environment is a core component of this task. With the rise of automotive 4D millimeter-wave radar point cloud technology, enabling autonomous vehicles to perceive their driving environment is another important task, building on the two main challenges of increasing point cloud quantity and improving point cloud quality. Currently, deep learning-based convolutional neural networks (CNNs), probabilistic graph-based methods (PGMs), real-time 3D map construction driven by lidar data, support vector machines (SVMs), clustering-based methods, and adaptive thresholding methods are all helping vehicles achieve driving area recognition. However, these methods still have room for improvement for 4D millimeter-wave radar point clouds, primarily because millimeter-wave radar point clouds are inferior to lidar point clouds, and both their quality and quantity directly impact the effectiveness of the aforementioned algorithms.

[0004] In order to solve the above technical problems, the present invention proposes a real-time free space detection method, device and computer-readable storage medium based on a quasi-density tree structure, which can realize target recognition in the environment and free space recognition at the same time. Summary of the Invention

[0005] The present invention provides a free space real-time detection method, device and computer-readable storage medium based on a quasi-density tree structure, which can realize target recognition in an environment and identify free space at the same time.

[0006] The embodiment of the first aspect of the present invention provides a free space real-time detection method based on a quasi-density tree structure, comprising the following steps: extracting point cloud data at a specific spatial height according to the vehicle height, and preprocessing the point cloud data, wherein the preprocessing includes gridding the point cloud data; accumulating multiple frames of data according to coordinate transformation rules; establishing a distance dictionary and an angle dictionary for each grid node relative to the center of the coordinate system, and establishing a subordinate point cloud dictionary for each grid node; establishing a dynamic matrix M s And the static matrix M m , the dynamic matrix and the static matrix have the same size as the coordinate plane after gridding; establish the node density matrix M d and the node occupancy matrix M e :The node density matrix and the node occupancy matrix are the same size as the coordinate plane after gridding; evaluate the probability of target existence at each node; obtain the dynamic target existence probability matrix and the static target existence probability matrix, and determine the comprehensive score matrix based on the dynamic target existence probability matrix and the static target existence probability matrix; and standardize the comprehensive score matrix to obtain the final comprehensive score matrix; set a numerical critical value for the final comprehensive score matrix to obtain the grid nodes where the dynamic targets are located and the grid nodes where the static targets are located; use the distance dictionary and the angle dictionary to compare and select the nearest grid node closest to the center of the coordinate system in each 1° area in the 360° direction, and compare the point closest to the coordinate center in the subordinate point cloud of the nearest grid node according to the subordinate point cloud dictionary, draw the drivable area image according to the coordinates of the point and output it.

[0007] Preferably, the point cloud data includes three-dimensional coordinates x, y, and z, a signal-to-noise ratio (SNR), and Doppler information; the gridding process specifically includes the following steps: gridding the x and y coordinate values ​​according to the analysis accuracy; calculating the target point and the vehicle speed and estimating whether it is a stationary target or a moving target; assigning a value to the signal-to-noise ratio (SNR), assigning a value of 1 if it is greater than a specific value, and assigning a value of 0 if it is less than or equal to the specific value.

[0008] Preferably, in the step of accumulating multiple frames of data according to the coordinate conversion rule, the coordinate conversion formula adopted by the coordinate conversion rule is as follows:

[0009]

[0010] Among them, x and y are the point cloud coordinates before conversion, x′ and y′ are the point cloud coordinates after conversion, and d is the distance traveled by the vehicle in the time interval between two frames of data. is the vehicle’s turning angle during the time interval between two frames of data.

[0011] Preferably, the steps of establishing a distance dictionary and an angle dictionary for each grid node relative to the center of the coordinate system, and establishing a subordinate point cloud dictionary for each grid node specifically include the following sub-steps: dividing the point cloud space into a uniform grid space centered on the vehicle according to the analysis accuracy, or gridding the point cloud x and y coordinates to obtain a grid system under a bird's-eye view; establishing a distance dictionary and an angle dictionary for each grid node relative to the center of the coordinate system; and establishing a subordinate point cloud dictionary for each grid node based on the grid nodes corresponding to the gridded x and y coordinates in the distance dictionary and the angle dictionary.

[0012] Preferably, a dynamic matrix M is established s And the static matrix M m The specific steps include the following process: according to the point cloud data and the relative speed information of the vehicle, the node value of the dynamic point cloud is marked as 1 in the dynamic matrix, otherwise it is 0; the node value of the static point cloud is marked as 1 in the static matrix, otherwise it is 0; establish the node density matrix M d and the node occupancy matrix M e The steps specifically include the following process: the number of subordinate point clouds at each node is set to the value of the corresponding position of the node density matrix through the node density matrix, and the node occupancy matrix is ​​used to count the number of point clouds with an SNR value of 1 in the subordinate point clouds of each node after gridding.

[0013] Preferably, in the step of evaluating the existence probability of each node target, the evaluation is performed according to the following formula:

[0014] [M e / M d M e ][w1 w2] T =M score ,

[0015] Among them, M e / M d is the numerical division at the corresponding position between the two matrices, w1 and w2 are weight values, the former corresponds to M e / M d The probability of target existence at the node level, the latter corresponds to M e Probability of target existence at the point cloud level.

[0016] Preferably, the dynamic target existence probability matrix M score-m And the static target existence probability matrix M score-s It is obtained by the following calculation:

[0017] [M s ;M m ]◎M score =[M score-s ;M score-m ],

[0018] Where ◎ represents the multiplication between the corresponding values ​​of the matrix; and in the step of determining the comprehensive score matrix based on the dynamic target existence probability matrix and the static target existence probability matrix: convolution calculation is performed on the dynamic target existence probability matrix and the static target existence probability matrix. The calculation formula is as follows:

[0019] conv([M score-s ;M score-m ],M weight )=[M vote-s ;M vote-m ],

[0020] Get the comprehensive score matrix M determined by the votes of the node and its neighbors vote-s and M vote-m , where M weight is the voting weight matrix of the node and its neighbors; and the comprehensive score matrix is ​​standardized to obtain the final comprehensive score matrix step, according to the expression

[0021] norm([M vote-s ;M vote-m ]◎[M s ;M m ])=[M vote-ns ;M vote-nm ]

[0022] Confirm and standardize the comprehensive score matrix to obtain the final comprehensive score matrix M with a final score between [0,1] vote-ns and M vote-nm .

[0023] Preferably, a numerical critical value is set for the final comprehensive score matrix. In the step of obtaining the grid node where the dynamic target is located and the grid node where the static target is located, the elements in the final comprehensive score matrix that are greater than the critical value are set to 1, and vice versa, and then the coordinates of all elements with a value of 1 in the matrix are obtained to obtain the grid node where the dynamic target is located and the grid node where the static target is located.

[0024] An embodiment of the second aspect of the present invention also provides a free space real-time detection device based on a quasi-density tree structure, which includes a memory and a processor; wherein the memory is used to store executable program code; the processor is used to read the executable program code stored in the memory to execute the free space real-time detection method based on the quasi-density tree structure.

[0025] An embodiment of the third aspect of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, a real-time free space detection method based on a quasi-density tree structure is implemented.

[0026] The present invention provides a real-time free space detection method, device and computer-readable storage medium based on a quasi-density tree structure, which can simultaneously identify targets in an environment and identify free spaces.

[0027] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0029] Figure 1 A schematic diagram illustrating boundary point selection according to a free space real-time detection method based on a quasi-density tree structure according to an embodiment of the present invention is shown;

[0030] Figure 2 This is a structural diagram of an embodiment of a free space real-time detection device based on a quasi-density tree structure in this specification;

[0031] Figure 3 This is a structural diagram of an embodiment of a computer-readable storage medium of the free space real-time detection method based on a quasi-density tree structure of this specification. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0034] The following discussion provides multiple embodiments of the present invention. Although each embodiment represents a single combination of the invention, different embodiments of the present invention can be substituted or combined, and the present invention is also considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes A, B, and C, and another embodiment includes a combination of B and D, then the present invention should also be considered to include embodiments that include one or more of A, B, C, and D in all other possible combinations, even if such embodiments may not be explicitly described in the following text.

[0035] Figure 1FIG. 1 shows a schematic diagram of boundary point selection according to a free space real-time detection method based on a quasi-density tree structure according to an embodiment of the present invention. Figure 1 As shown, the free space real-time detection method based on the quasi-density tree structure provided by the embodiment of the present invention includes the following steps: extracting point cloud data at a specific space height according to the vehicle height, and preprocessing the point cloud data, the preprocessing including gridding the point cloud data; accumulating multiple frames of data according to the coordinate transformation rules; establishing a distance dictionary and an angle dictionary of each grid node relative to the center of the coordinate system, and establishing a subordinate point cloud dictionary of each grid node; establishing a dynamic matrix M s And the static matrix M m , the dynamic matrix and the static matrix have the same size as the coordinate plane after gridding; establish the node density matrix M d and the node occupancy matrix M e :The node density matrix and the node occupancy matrix are the same size as the coordinate plane after gridding; evaluate the probability of target existence at each node; obtain the dynamic target existence probability matrix and the static target existence probability matrix, and determine the comprehensive score matrix based on the dynamic target existence probability matrix and the static target existence probability matrix; and standardize the comprehensive score matrix to obtain the final comprehensive score matrix; set a numerical critical value for the final comprehensive score matrix to obtain the grid nodes where the dynamic targets are located and the grid nodes where the static targets are located; use the distance dictionary and the angle dictionary to compare and select the nearest grid node closest to the center of the coordinate system in each 1° area in the 360° direction, and compare the point closest to the coordinate center in the subordinate point cloud of the nearest grid node according to the subordinate point cloud dictionary, draw the drivable area image according to the coordinates of the point and output it.

[0036] Existing free space recognition algorithms rely on cameras, lidar point clouds, or sensor fusion, with various approaches. However, relying solely on 4D millimeter-wave radar point clouds to identify environmental free space faces the problem of low point cloud quality. The real-time free space detection method based on a quasi-density tree structure, provided in embodiments of the present invention, aims to simultaneously identify targets in the environment and free space. It uses 4D millimeter-wave radar point clouds to outline environmental targets, reducing data noise and enabling efficient identification of free space in the driving environment. It also reduces the algorithm's reliance on datasets while ensuring the stability and reliability of calculation results. It achieves faster and more efficient computation, enabling real-time point cloud data processing.

[0037] In the free space real-time detection method based on a quasi-density tree structure provided in an embodiment of the present invention, after obtaining point cloud data, the point cloud data is preprocessed. The point cloud data includes information such as three-dimensional coordinates x, y, and z, signal-to-noise ratio (SNR), and Doppler. Based on the vehicle's height, point cloud data at a specific spatial height (e.g., 1.5 meters) is extracted. The x and y coordinate values ​​of the data are gridded according to a specific analysis accuracy (e.g., 0.5 meters). The target point and the vehicle's speed are calculated to estimate whether it is a stationary or moving target. The SNR is converted to a 0-1 value, that is, a value greater than a specific value (e.g., 17) is set to 1, and a value less than a specific value is set to 0.

[0038] In the free space real-time detection method based on the quasi-density tree structure provided by the embodiment of the present invention, multiple frames of data are accumulated according to the following coordinate conversion formula:

[0039]

[0040] Here, x and y are the point cloud coordinates before conversion, x' and y' are the point cloud coordinates after conversion, and d is the distance traveled by the vehicle in the time interval between two frames of data. is the vehicle’s turning angle during the time interval between two frames of data.

[0041] The free space real-time detection method based on a quasi-density tree structure provided in an embodiment of the present invention includes a tree structure establishment step, in which the point cloud space is divided into a uniform grid space centered on the vehicle according to the aforementioned required analysis accuracy. Alternatively, only the x and y coordinates of the point cloud can be gridded, that is, a bird's-eye view (BEV) grid system is obtained, thereby establishing a dictionary of the distance and orientation (angle) of each grid node relative to the center of the coordinate system. At the same time, the x and y coordinates after gridding in the point cloud data preprocessing step will have corresponding grid nodes in the two dictionaries, so a subordinate point cloud dictionary of each grid node is established, that is, which gridded point cloud data points are on each node.

[0042] In the free space real-time detection method based on the quasi-density tree structure provided by the embodiment of the present invention, a dynamic matrix M is established. s , static matrix M m , node density matrix M d and the node occupancy matrix M eThe dynamic matrix and the static matrix have the same size as the coordinate plane after gridding. According to the relative speed information between the point cloud and the vehicle, the node value of the dynamic point cloud is marked as 1 in the dynamic matrix, otherwise it is 0. The node value of the static point cloud is marked as 1 in the static matrix, otherwise it is 0. The node density matrix and the node occupancy matrix have the same size as the coordinate plane after gridding. The number of subordinate point clouds at each node is set to the value of the corresponding position in the node density matrix, and the node occupancy matrix is ​​used to count the number of point clouds with an SNR value of 1 in the subordinate point clouds of each node after gridding.

[0043] In the free space real-time detection method based on a quasi-density tree structure provided by an embodiment of the present invention, in the step of evaluating the probability of the target existing at each node, the evaluation is performed according to the following formula:

[0044] [M e / M d M e ][w1 w2] T =M score ,

[0045] Among them, M e / M d is the numerical division at the corresponding position between the two matrices, w1 and w2 are weight values, the former corresponds to M e / M d The probability of target existence at the node level, the latter corresponds to M e The probability of target existence at the point cloud level. Dynamic target existence probability matrix M score-m And the static target existence probability matrix M score-s It is obtained by the following calculation:

[0046] [M s ;M m ]◎M score =[M score-s ;M score-m ],

[0047] Where ◎ represents the multiplication between the corresponding values ​​of the matrix; and in the step of determining the comprehensive score matrix based on the dynamic target existence probability matrix and the static target existence probability matrix: convolution calculation is performed on the dynamic target existence probability matrix and the static target existence probability matrix. The calculation formula is as follows:

[0048] conv([M score-s ;M score-m ],M weight )=[M vote-s ;M vote-m ],

[0049] Get the comprehensive score matrix M determined by the votes of the node and its neighbors vote-s and Mvote-m , where M weight is the voting weight matrix of the node and its neighbors; and the comprehensive score matrix is ​​standardized to obtain the final comprehensive score matrix step, according to the expression

[0050] norm([M vote-s ;M vote-m ]◎[M s ;M m ])=[M vote-ns ;M vote-nm ]

[0051] Confirm and standardize the comprehensive score matrix to obtain the final comprehensive score matrix M with a final score between [0,1] vote-ns and M vote-nm .

[0052] In the free space real-time detection method based on the quasi-density tree structure provided by the embodiment of the present invention, the final comprehensive score matrix M vote-ns and M vote-nm Set a numerical critical value (such as 0.5), set the elements greater than the critical value to 1, and vice versa to 0, and then obtain the coordinates of all elements with a value of 1 in the matrix to obtain the grid nodes where the dynamic and static targets are located.

[0053] like Figure 1 As shown, in the free space real-time detection method based on the quasi-density tree structure provided by the embodiment of the present invention, the distance and orientation (angle) dictionary of the grid nodes relative to the center of the coordinate system is used to compare and select the grid nodes closest to the coordinate center in the 360-degree direction (such as Figure 1 ), compare the subordinate point clouds of these grid nodes in the 360-degree direction (such as Figure 1 The point closest to the coordinate center in the second figure) Figure 1 Based on the coordinates of the points obtained above, draw an image of the drivable area (free space) and output it.

[0054] The real-time free space detection method based on a quasi-density tree structure provided by the embodiments of the present invention introduces point cloud density features, effectively identifying 4D millimeter-wave radar point cloud noise and facilitating free space identification. The algorithm has a wide effective parameter range, making parameter adjustment easy. The algorithm process is easy to implement using parallel computing, thereby accelerating the calculation process.

[0055] Figure 2 This is a structural diagram of an embodiment of a free space real-time detection device based on a quasi-density tree structure in this specification. Figure 2, which shows a schematic structural diagram of a freespace real-time detection device 300 based on a quasi-density tree structure suitable for implementing an embodiment of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 2 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0056] like Figure 2 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0057] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 2 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0058] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0059] Figure 3 This is a structural diagram of an embodiment of a computer-readable storage medium of a free space real-time detection method based on a quasi-density tree structure in this specification. Figure 3 As shown, a computer-readable storage medium 40 according to an embodiment of the present disclosure stores non-transitory computer-readable instructions 41. When the non-transitory computer-readable instructions 41 are executed by a processor, all or part of the steps of the free space real-time detection method based on a quasi-density tree structure according to the aforementioned embodiments of the present disclosure are executed.

[0060] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0061] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0062] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: constructs a basic page, and the page code of the basic page is used to build the environment required for the operation of the business page and / or implement the same workflow abstracted in the same business scenario; constructs one or more page templates, and the page templates are used to provide code templates for implementing business functions in the business scenario; based on the corresponding page templates, the final page code of each page of the business scenario is generated through code conversion of the specific functions of each page of the business scenario; the generated final page code of each page is merged into the page code of the basic page to generate the code of the business page.

[0063] Alternatively, the computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: constructs a basic page, the page code of the basic page is used to build the environment required for the operation of the business page and / or implement the same workflow abstracted from similar business scenarios; constructs one or more page templates, the page templates are used to provide code templates for implementing business functions in business scenarios; based on the corresponding page templates, the final page code of each page of the business scenario is generated through code conversion of the specific functions of each page of the business scenario; the generated final page code of each page is merged into the page code of the basic page to generate the code of the business page.

[0064] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0065] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0066] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0067] The present invention provides a real-time free space detection method, device and computer-readable storage medium based on a quasi-density tree structure, which can simultaneously identify targets in an environment and identify free spaces.

[0068] In the present invention, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "plurality" refers to two or more, unless expressly limited otherwise. Terms such as "installed," "connected," "connected," and "fixed" should be interpreted broadly. For example, "connected" can mean a fixed connection, a detachable connection, or an integral connection; "connected" can mean a direct connection or an indirect connection through an intermediary. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0069] In the description of the present invention, it should be understood that the terms "upper" and "lower" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or unit referred to must have a specific direction, be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on the present invention.

[0070] Throughout this specification, terms such as "one embodiment," "some embodiments," and "specific embodiments" mean that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0071] The above are only some embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A free space real-time detection method based on a quasi-density tree structure, characterized in that: The steps include: Extracting point cloud data at a specific spatial height according to the vehicle height, and preprocessing the point cloud data, wherein the preprocessing includes gridding the point cloud data; Accumulate multiple frames of data according to coordinate transformation rules; Establish a distance dictionary and an angle dictionary for each grid node relative to the center of the coordinate system, and establish a subordinate point cloud dictionary for each grid node; Establish dynamic matrix M s And the static matrix M m , the dynamic matrix and the static matrix have the same size as the coordinate plane after gridding; Establish node density matrix M d and the node occupancy matrix M e : The node density matrix and the node occupancy matrix have the same size as the coordinate plane after gridding; Evaluate the probability of existence of each node target; Obtaining a dynamic target existence probability matrix and a static target existence probability matrix, and determining a comprehensive score matrix based on the dynamic target existence probability matrix and the static target existence probability matrix; and normalizing the comprehensive score matrix to obtain a final comprehensive score matrix; Setting a numerical critical value for the final comprehensive score matrix to obtain the grid nodes where the dynamic targets are located and the grid nodes where the static targets are located; Using the distance dictionary and the angle dictionary, compare and filter out the nearest grid node closest to the center of the coordinate system in each 1° area in the 360° direction, and compare the point closest to the coordinate center in the subordinate point cloud of the nearest grid node according to the subordinate point cloud dictionary, draw the drivable area image according to the coordinates of the point and output it.

2. The free space real-time detection method based on quasi-density tree structure according to claim 1 is characterized in that: The point cloud data includes three-dimensional coordinates x, y, z, signal-to-noise ratio (SNR), and Doppler information. The gridding process specifically includes the following steps: Grid the x and y coordinate values ​​according to the analysis accuracy; Calculate the target point and the vehicle's speed and estimate whether it is a stationary target or a moving target; The signal-to-noise ratio (SNR) is assigned a value of 1 if it is greater than a specific value, and a value of 0 if it is less than or equal to a specific value.

3. The free space real-time detection method based on quasi-density tree structure according to claim 2 is characterized in that: In the step of accumulating multiple frames of data according to the coordinate conversion rule, the coordinate conversion formula used by the coordinate conversion rule is as follows: Among them, x and y are the point cloud coordinates before conversion, x′ and y′ are the point cloud coordinates after conversion, and d is the distance traveled by the vehicle in the time interval between two frames of data. is the vehicle’s turning angle during the time interval between two frames of data.

4. The free space real-time detection method based on quasi-density tree structure according to claim 3 is characterized in that: The steps of establishing a distance dictionary and an angle dictionary of each grid node relative to the center of the coordinate system, and establishing a subordinate point cloud dictionary of each grid node specifically include the following sub-steps: Dividing the point cloud space into a uniform grid space centered on the vehicle according to the analysis accuracy, or performing grid division on the x and y coordinates of the point cloud to obtain a grid system under a bird's-eye view; Establish a distance dictionary and an angle dictionary for each grid node relative to the center of the coordinate system; According to the grid nodes corresponding to the gridded x and y coordinates in the distance dictionary and the angle dictionary, a subordinate point cloud dictionary of each grid node is established.

5. The free space real-time detection method based on quasi-density tree structure according to claim 4 is characterized by: The dynamic matrix M is established s And the static matrix M m The steps specifically include the following process: according to the relative speed information between the point cloud data and the vehicle, the node value of the dynamic point cloud is marked as 1 in the dynamic matrix, otherwise it is marked as 0; the node value of the static point cloud is marked as 1 in the static matrix, otherwise it is marked as 0; The node density matrix M is established d and the node occupancy matrix M e The steps specifically include the following process: the number of subordinate point clouds at each node is set to the value of the corresponding position of the node density matrix through the node density matrix, and the node occupancy matrix is ​​used to count the number of point clouds with an SNR value of 1 in the subordinate point clouds of each node after gridding.

6. The free space real-time detection method based on quasi-density tree structure according to claim 5, characterized in that: In the step of evaluating the existence probability of each node target, the evaluation is performed according to the following formula: [M e / M d M e ][w1 w2] T =M score , Among them, M e / M d is the numerical division at the corresponding position between the two matrices, w1 and w2 are weight values, the former corresponds to M e / M d The probability of target existence at the node level, the latter corresponds to M e Probability of target existence at the point cloud level.

7. The free space real-time detection method based on quasi-density tree structure according to claim 6, characterized in that: Dynamic target existence probability matrix M score-m And the static target existence probability matrix M score-s It is obtained by the following calculation: [M s ;M m ]◎M score =[M score-s ;M score-m ], Where ◎ represents the multiplication between the corresponding positions of the matrix; In the step of determining the comprehensive score matrix based on the dynamic target existence probability matrix and the static target existence probability matrix: performing convolution calculation on the dynamic target existence probability matrix and the static target existence probability matrix, the calculation formula is as follows: conv([M score-s ;M score-m ],M weight )=[M vote-s ;M vote-m ], Get the comprehensive score matrix M determined by the votes of the node and its neighbors vote-s and M vote-m , where M weight The voting weight matrix for the node and its neighbors; In the step of normalizing the comprehensive score matrix to obtain the final comprehensive score matrix, according to the expression norm([M vote-s ;M vote-m ]◎[M s ;M m ])=[M vote-ns ;M vote-nm ] Confirm and standardize the comprehensive score matrix to obtain the final comprehensive score matrix M with a final score between [0,1] vote-ns and M vote-nm .

8. The free space real-time detection method based on a quasi-density tree structure according to any one of claims 1 to 7, characterized in that: In the step of setting a numerical critical value for the final comprehensive score matrix and obtaining the grid nodes where the dynamic targets and the static targets are located, the elements in the final comprehensive score matrix that are greater than the critical value are set to 1, and vice versa, and then the coordinates of all elements with a value of 1 in the matrix are obtained to obtain the grid nodes where the dynamic targets and the static targets are located.

9. A free space real-time detection device based on a quasi-density tree structure, comprising a memory and a processor; wherein, The memory is used to store executable program code; The processor is configured to read the executable program code stored in the memory to execute the free space real-time detection method based on a quasi-density tree structure according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the free space real-time detection method based on a quasi-density tree structure according to any one of claims 1 to 8 is implemented.

Images