Point cloud detection method and device, storage medium and electronic equipment

Abstract

The invention provides a point cloud detection method and device, a storage medium and electronic equipment. The point cloud detection method comprises the following steps: carrying out regional division on a point cloud detection spectrum to obtain spectrum regions, and obtaining a target type of each spectrum region; determining the point cloud detection density of the frequency spectrum region according to the target type; and performing point cloud detection based on the point cloud detection density. According to the invention, the performance of point cloud detection can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of point cloud detection, and relates to a point cloud detection method, and in particular to a point cloud detection method, device, storage medium and electronic device. Background Art

[0002] Point cloud detection has been widely used in computer vision, robot navigation, and autonomous driving. Point cloud detection is usually based on point cloud data collected by high-precision sensors (such as lidar, depth camera, millimeter wave radar, etc.). By analyzing and processing point cloud data, it can identify and extract target objects and their features in space, and realize the perception and understanding of the environment. However, the output effect of existing point cloud detection technology is not ideal, and there are still some problems that need to be improved. Summary of the invention

[0003] The embodiments of the present application provide a point cloud detection method, device, storage medium and electronic device for improving the performance of point cloud detection.

[0004] In a first aspect, an embodiment of the present application provides a point cloud detection method, which includes: dividing the point cloud detection spectrum into regions to obtain spectrum regions, and acquiring a target type for each of the spectrum regions; determining a point cloud detection density of the spectrum region according to the target type; and performing point cloud detection based on the point cloud detection density.

[0005] In an implementation of the first aspect, the point cloud detection spectrum is divided into regions to obtain spectrum regions, and obtaining the target type of each spectrum region includes: obtaining a post-processed target classification result; obtaining target dimension information corresponding to each type of target in the target classification result, wherein the target dimension is an information dimension that can be represented by the point cloud detection spectrum; projecting each type of target in the target classification result to the point cloud detection spectrum according to the target dimension information; and obtaining the spectrum region and its target type according to the projection result.

[0006] In an implementation of the first aspect, dividing the point cloud detection spectrum into regions to obtain spectrum regions, and obtaining the target type of each of the spectrum regions includes: dividing the point cloud detection spectrum into regions to obtain the spectrum regions and obtaining the target type of each of the spectrum regions using a spectrum division and classification model, wherein the spectrum division and classification model is a trained deep learning model.

[0007] In an implementation of the first aspect, the point cloud detection spectrum is divided into regions to obtain spectrum regions, and obtaining the target type of each spectrum region includes: obtaining the lateral distance of each reflection point in the reference point cloud data; obtaining the point cloud proportion of each lateral distance by a statistical method; obtaining corresponding target dimension information according to the lateral distance and radial distance of the stationary point, wherein the target dimension is an information dimension that can be represented by the point cloud detection spectrum, and the stationary point is a reflection point corresponding to the lateral distance in which the point cloud proportion is greater than a ratio threshold; and obtaining the spectrum region corresponding to the stationary target and / or the moving target according to the target dimension information.

[0008] In an implementation of the first aspect, determining the point cloud detection density of the spectrum region according to the target type includes: determining the point cloud detection density of the spectrum region corresponding to the target type according to a detection accuracy requirement of the target type.

[0009] In an implementation of the first aspect, performing point cloud detection based on the point cloud detection density includes: for a first spectral region where the detection density is greater than a reference density, point cloud detection is performed using a strategy of shielding local peak detection and / or removing constant false alarm noise interference, and the reflection points in the first spectral region whose energy is greater than an energy threshold and whose energies between adjacent resolution units satisfy a ratio condition are taken as output points; for a second spectral region where the detection density is not greater than the reference density, point cloud detection is performed using a local peak detection and / or constant false alarm detection strategy.

[0010] In an implementation of the first aspect, the point cloud detection method is applied to a millimeter wave radar system whose point cloud detection process is fixed and does not support local configuration detection strategies, and point cloud detection based on the point cloud detection density includes: if the point cloud detection density of the frequency spectrum area of ​​interest is greater than the reference density, then: using the millimeter wave radar system to perform a first point cloud detection to obtain a point cloud detection result; based on the point cloud detection result, obtaining a detection point located in the frequency spectrum area of ​​interest as a point of interest; obtaining surrounding resolution units of the point of interest; adding the surrounding resolution units that meet preset conditions to an output point cloud list of the first point cloud detection; if the point cloud detection density of the frequency spectrum area of ​​interest is not greater than the reference density, then: using the millimeter wave radar system to perform a second point cloud detection to obtain a point cloud detection result.

[0011] In a second aspect, an embodiment of the present application provides a point cloud detection device, comprising: a spectrum division and classification module, for dividing the point cloud detection spectrum into regions to obtain spectrum regions, and obtaining the target type of each spectrum region; a detection density determination module, for determining the point cloud detection density of the spectrum region according to the target type; and a point cloud detection module, for performing point cloud detection based on the point cloud detection density.

[0012] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the point cloud detection method described in any one of the first aspects of the embodiments of the present application is implemented.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory storing a computer program; and a processor communicatively connected to the memory, for executing the point cloud detection method described in any one of the first aspects of the embodiments of the present application when the computer program is called.

[0014] The embodiment of the present application divides and classifies the point cloud detection spectrum into local areas, obtains multiple spectrum areas corresponding to different target types, and determines the point cloud detection density of each spectrum area. Point cloud detection is performed based on the determined point cloud detection density, and the focus target can be freely switched according to user needs, laying an important foundation for the system to output ideal point cloud detection effects.

[0015] Compared with the prior art, the embodiment of the present application can improve the integration of the point cloud detection stage with the actual scene and post-processing, and improve the flexibility of the millimeter wave radar system for multiple application scenarios. In addition, the embodiment of the present application has no special requirements for the front-end waveform design, signal processing flow, etc., and has high applicability.

[0016] The embodiment of the present application configures appropriate point cloud detection strategies for different spectrum regions based on the results of spectrum division and classification, and outputs a point cloud density that meets user needs. Detection strategies such as shielding local peak detection and / or removing constant false alarm noise interference are implemented for the first spectrum region of interest to solve the problem of low-density point clouds caused by inappropriate application of peak detection strategies to surface targets such as vehicles. For the second spectrum region that has relatively low point cloud density requirements and is prone to false alarms, detection strategies such as local peak detection and / or constant false alarm detection are adopted to control the point cloud density and false alarm probability. Therefore, the embodiment of the present application can greatly suppress the false alarm probability while enhancing the point cloud density of the target of interest.

[0017] For systems that do not support local configuration difference point cloud detection strategies, the embodiments of the present application provide a method for adding new valid points based on the original process detection points, which greatly improves the feasibility of the implementation of the solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a structural schematic diagram of the point cloud detection system.

[0019] Figure 2 Shown is a flow chart of a point cloud detection method provided in an embodiment of the present application.

[0020] Figure 3Shown is a flow chart of an implementation method for dividing spectrum regions and determining target types in an embodiment of the present application.

[0021] Figure 4 Shown is a flowchart of another method for dividing spectrum regions and determining target types in an embodiment of the present application.

[0022] Figure 5 It shows a flow chart of adding valid points based on the original process detection points in the embodiment of the present application.

[0023] Figure 6 Shown is an example diagram of the point cloud detection results obtained in the embodiments of the present application.

[0024] Figure 7 Shown is a schematic diagram of the structure of a point cloud detection device provided in an embodiment of the present application.

[0025] Figure 8 Shown is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0026] Component number description

[0027] 700 Point Cloud Detection Device

[0028] 701 Spectrum Allocation and Classification Module

[0029] 702 Detection density determination module

[0030] 703 Point Cloud Detection Module

[0031] 800 Electronic equipment

[0032] 801 Processor

[0033] 802 Memory

[0034] 8021 Operating System

[0035] 8022 Applications

[0036] 803 Network Interface

[0037] 804 bus system

[0038] 805 User Interface

[0039] Steps S21 to S23

[0040] Steps S31 to S34

[0041] Steps S41 to S44

[0042] Steps S51 to S54 DETAILED DESCRIPTION

[0043] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0044] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0045] In the embodiments of the present application, the words "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0046] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0047] Figure 1 The diagram shows the structure of a point cloud detection system based on millimeter wave radar in some technical solutions. Figure 1 As shown, the point cloud detection system in these technical solutions includes a millimeter wave radar sensor, a signal processing unit and a point cloud processing unit.

[0048] The millimeter wave radar sensor includes an antenna array, an RF front-end (RF Front-End) and an intermediate frequency processor (IF). The antenna array is used to transmit and receive millimeter wave signals. The RF front-end includes a transmitting and receiving module for generating and processing high-frequency millimeter wave signals. The transmitter is used to modulate the low-frequency signal into a high-frequency signal and transmit it through the antenna, and the receiver is used to capture the signal reflected from the target and perform down-conversion processing. The intermediate frequency processor is used to convert the received high-frequency signal into a lower frequency signal for subsequent digitization and processing.

[0049] The signal processing unit includes an analog-to-digital converter (ADC) and a digital signal processor (DSP) / field-programmable gate array (FPGA). The analog-to-digital converter is used to convert analog signals into digital signals. The digital signal processor (DSP) / field-programmable gate array (FPGA) is used to further process the digitized signals, such as fast Fourier transform (FFT), filtering, time domain and frequency domain analysis, etc., to generate preliminary distance, speed, angle and other information.

[0050] The point cloud processing unit includes a microcontroller unit (MCU) and a storage unit. The microcontroller is used to further process the point cloud data from the DSP / FPGA, such as clustering, classification, and tracking of the point cloud. The storage unit is used to store point cloud data, intermediate results, and processed information.

[0051] In some implementations, the point cloud detection system may further include a power module and a communication module. The power module is used to provide the required power for the entire system to ensure that each hardware component works stably. The communication module is used to transmit the processed point cloud data and other radar information to an external system.

[0052] It should be noted that Figure 1 The point cloud detection system based on millimeter wave radar is introduced as an example, but the present application is not limited to this.

[0053] In the point cloud detection stage, the same point cloud detection strategy is usually used for the full-band reflection points in the distance dimension (fast time) and the Doppler dimension (slow time), including signal accumulation method, constant false alarm detection method, peak detection method, detection threshold setting, etc., to reduce the calculation pressure and storage pressure of the point cloud detection system and simplify the processing flow. However, the expected detection effects of different targets are often different. From the perspective of system application, it is usually hoped that a dense point cloud is used for detection of moving targets in order to depict their contours, while for stationary targets (such as guardrails), considering factors such as system storage pressure, it is usually hoped that a certain point cloud density is abandoned and its position and approximate shape can be output. From the performance of the signal in the frequency domain, the complexity of the frequency domain signals of different targets in the same frequency range is similar. For example, in the relatively close range area (relatively low-frequency area) of the radar, for moving targets, there are problems such as large-area micro-motion characteristics, spectrum expansion caused by non-ideal coherence, and weak energy of target reflection points. For stationary targets, due to their irregular structure, material, area, etc., there are problems such as the signal spanning a wide frequency range and the signal is easily interfered by multipath to produce false targets. At this time, if a set of point cloud detection strategies is used for all areas, it is inevitable to make a choice between missed detection of moving targets and false alarm of stationary targets, resulting in either the point cloud density of moving targets not meeting the requirements or a large number of false alarm points being output, making it impossible to achieve good detection performance.

[0054] At least to address the above problems, the present application provides a point cloud detection method. The technical solution in the present application will be described in detail below in conjunction with the accompanying drawings in the present application.

[0055] Figure 2 The flowchart of the point cloud detection method provided by the embodiment of the present application is shown. Figure 2 As shown, the point cloud detection method provided in the embodiment of the present application includes the following steps S21 to S23.

[0056] S21, divide the point cloud detection spectrum into regions to obtain spectrum regions, and obtain the target type of each spectrum region. Step S21 can locally divide the point cloud detection spectrum and confirm the target type of different spectrum regions, for example, confirm which spectrum region belongs to a stationary target, that is, a stationary obstacle in the environment, such as a guardrail; confirm which spectrum region belongs to a moving target, such as a vehicle, a pedestrian, etc.

[0057] It should be understood that the embodiment of the present application does not limit the shape of the divided spectrum region, as long as it is ensured that the frequency components contained in each spectrum region obtained after the division are from the same category or the same independent target as much as possible.

[0058] S22, determining a point cloud detection strategy for the spectrum region according to the target type. In some implementations, the point cloud detection strategy includes point cloud detection density. In these implementations, the point cloud detection density of the spectrum region can be determined according to the target type.

[0059] S23, performing point cloud detection based on the point cloud detection strategy.

[0060] In some implementations, point cloud detection strategies for all spectral regions may be determined in step S22 , and point cloud detection may be performed according to the point cloud detection strategies for all spectral regions in step S23 .

[0061] In some implementations, a point cloud detection strategy for the spectral region of interest may be determined in step S22 , and point cloud detection may be performed according to the point cloud detection strategy for the spectral region of interest in step S23 .

[0062] As described above, in the embodiments of the present application, the point cloud detection spectrum can be divided and classified into local areas, multiple spectrum areas corresponding to different target types can be obtained, and the point cloud detection strategy for each spectrum area can be determined. Point cloud detection is performed based on the determined point cloud detection strategy, and the focus target can be switched freely according to user needs, laying an important foundation for the system to output an ideal point cloud detection effect. Compared with the prior art, the embodiments of the present application can improve the integration of the point cloud detection stage with the actual scene and post-processing, and improve the flexibility of the millimeter wave radar system to correspond to multiple application scenarios. In addition, the embodiments of the present application have no special requirements for the front-end waveform design, signal processing flow, etc., and have high applicability.

[0063] See also Figure 3 In some implementations, the point cloud detection spectrum is divided into regions to obtain spectrum regions, and an implementation method for obtaining the target type of each spectrum region includes the following steps S31 to S34.

[0064] S31, obtaining a post-processed target classification result, wherein post-processing refers to tracking the point cloud information in time and obtaining point clouds contained in different types of targets through clustering.

[0065] S32, obtaining target dimension information corresponding to each type of target in the target classification result, wherein the target dimension is an information dimension that can be represented by a point cloud detection spectrum, such as radial distance, Doppler velocity and / or azimuth angle, etc.

[0066] S33, projecting each type of target in the target classification result to a point cloud detection spectrum according to the target dimension information.

[0067] S34, obtaining the spectrum area and its target type according to the projection result. Specifically, after the projection, each type of target corresponds to a spectrum area, and the spectrum area division can be achieved in this way. In addition, since the types of various targets are known, the target type corresponding to the projection area can also be determined after projecting to the point cloud detection spectrum. For example, if the target is a track, and the spectrum area where the track is located after being projected to the point cloud detection spectrum is A, then it can be determined that the target type of spectrum area A is the track. For another example, when the target is a guardrail, and the spectrum area where the guardrail is located after being projected to the point cloud detection spectrum is B, then it can be determined that the target type of spectrum area B is the guardrail.

[0068] Through the above steps S31 to S34, the point cloud detection spectrum can be divided into a stationary target area and / or a moving target area, but the present application is not limited thereto.

[0069] Exemplarily, when the target is a track, the track information such as the position, speed, and direction of travel can be used, combined with the radar installation information and motion information, through the speed projection formula, azimuth calculation formula, and radial calculation formula, etc., to calculate the corresponding target dimension information, and then the track can be projected to the point cloud detection spectrum.

[0070] Exemplarily, the post-processing results of the previous frame or the previous frames can be used to implement the spectrum area division and classification of the current frame. Since the radar and the target may move relative to each other, in the embodiment of the present application, the target can be projected into the point cloud detection spectrum through motion compensation to improve the accuracy of the scheme. For example, the target motion model can be used between frames to extrapolate relevant parameters and add a certain protection margin to the parameters according to the target type. In this way, the effectiveness of the scheme can be effectively improved.

[0071] For example, the azimuth angle calculation formula and the radial calculation formula can be used to convert the PX (lateral distance relative to the vehicle body coordinate system) and PY (longitudinal distance relative to the vehicle body coordinate system) of the center point of the track into the lateral and longitudinal distances of the radar coordinate system, and the trigonometric geometry relationship can be used for calculation to realize the projection of the azimuth angle and radial distance.

[0072] For example, the Doppler velocity range corresponding to the track can be obtained using the velocity projection formula to achieve velocity projection. Specifically, the Doppler velocity V of the center point can be calculated using the track center point information. DopplerTrkBox , as shown in Equation 1:

[0073] V DopplerTrkBox =V TrkHeading *cos(β-θ)-[-V h *cos(θ)-V yawrate ], formula 1;

[0074] Among them, VTrkHeading V represents the speed of the track relative to the ground in its direction of travel, which can be calculated from its lateral and longitudinal speeds. β represents the direction of travel of the track, θ represents the azimuth angle of the track relative to the vehicle body coordinate system, and the two items in the brackets represent the projection components of the vehicle speed on the track azimuth angle θ. h Indicates the speed of the vehicle (radar-equipped vehicle), V yawrate It represents the velocity component caused by the steering angular velocity of the vehicle in the direction of the track azimuth angle θ.

[0075] Get the Doppler velocity V of the center point DopplerTrkBox After that, the Doppler velocity range of the entire track can be obtained by taking a certain redundancy before and after, that is, [V DopplerTrkBox -Δv1,V DopplerTrkBox +Δv2], where Δv1 and Δv2 represent the redundancy taken in the Doppler dimension.

[0076] Exemplarily, information of any one or more points on the track may be acquired to calculate the Doppler velocity range of the track.

[0077] For example, the Doppler velocity range of the point cloud belonging to the track may be recorded during the track generation stage.

[0078] Exemplarily, the azimuth angle θ of each reflection point on the stationary target relative to the vehicle body coordinate system may be acquired according to the stationary target information, and the bracketed portion in Formula 1 may be used to calculate the Doppler velocity of each reflection point on the stationary target.

[0079] By way of example, the Doppler velocity range of the point cloud belonging to the stationary object may be recorded during the generation phase.

[0080] It should be noted that the above only lists several exemplary ways of obtaining the Doppler velocity range, but the present application is not limited thereto.

[0081] In some implementations, dividing the point cloud detection spectrum into regions to obtain spectrum regions, and obtaining the target type of each spectrum region includes: dividing the point cloud detection spectrum into regions to obtain spectrum regions and obtaining the target type of each spectrum region using a spectrum division and classification model, wherein the spectrum division and classification model is a trained deep learning model. The point cloud detection spectrum can be divided into a stationary target region and a moving target region by the spectrum division and classification model, and the point cloud detection spectrum can also be divided into an area of ​​interest and an area of ​​non-interest, but the present application is not limited thereto.

[0082] See also Figure 4 In some implementations, dividing the point cloud detection spectrum into regions to obtain spectrum regions, and obtaining the target type of each spectrum region includes the following steps S41 to S44.

[0083] S41, obtain the lateral distance of each reflection point in the reference point cloud data. Specifically, stationary targets usually appear in a large area and in a regular arrangement, such as guardrails on both sides of the road. The point cloud attributes of stationary targets have the characteristics of relatively dense emission points and similar lateral distances (compared to millimeter wave radars), while the point cloud attributes of moving targets do not have the above characteristics. Based on this, in the embodiment of the present application, the point cloud can be roughly screened before the effective point cloud detection, and the coarse screened point cloud data or the adjacent historical effective point cloud data can be used as reference point cloud data to calculate relevant parameters to obtain the parameters of the stationary target. In this process, since the absolute speed of the reflection point from the stationary target is zero, its position on the spectrum can be uniquely confirmed by the radar installation information, the vehicle motion information and the reflection point angle, wherein the radar installation information and the vehicle motion information are known and obtainable information, and the reflection point angle can be calculated by the Doppler velocity and the vehicle speed and other vehicle body information, and then the radial distance and azimuth angle can be used to obtain the lateral distance of each reflection point.

[0084] S42, using a statistical method to obtain the percentage of point clouds at each lateral distance. For example, a histogram, a probability density function, etc. can be used to obtain the percentage of point clouds distributed at different lateral distances in the environment, where the percentage of point clouds is used to measure the ratio of the number of point clouds at different lateral distances.

[0085] S43, obtaining corresponding target dimension information according to the lateral distance and radial distance of the stationary point, wherein the target dimension is the information dimension that can be represented by the point cloud detection spectrum, and the stationary point refers to the reflection point corresponding to the lateral distance whose point cloud ratio is greater than the ratio threshold, and the above ratio threshold can be determined according to actual needs or experience.

[0086] S44, obtaining the spectral region corresponding to the stationary target and / or the moving target according to the target dimension information. Specifically, the reflection points of the stationary target have the characteristics of being relatively dense and having similar lateral distances. Therefore, it can be considered that the lateral positions where a large point cloud ratio appears (that is, the lateral distance where the point cloud ratio is greater than the ratio threshold) are the positions where the stationary target actually exists, and the reflection points corresponding to these positions are the stationary points. Based on this, the lateral distance value corresponding to the stationary point and its radial distance can be converted into the target dimension information, and then the spectral region corresponding to the stationary target can be obtained on the spectrum, and the area other than that can be used as the spectral region corresponding to the moving target.

[0087] In some implementations, determining the point cloud detection density of the spectrum region according to the target type includes: determining the point cloud detection density of the spectrum region corresponding to the target type according to a detection accuracy requirement of the target type.

[0088] Exemplarily, for a moving target of interest, the determined point cloud density should be able to describe the contour of the moving target.

[0089] Exemplarily, for a stationary object, the determined point cloud density should be able to describe the position and basic shape of the stationary object.

[0090] In some implementations, point cloud detection based on point cloud detection density includes: for the first spectrum region where the detection density is greater than the reference density, that is, the spectrum region of interest where the point cloud density needs to be increased additionally, point cloud detection can be performed using a strategy of shielding local peak detection and / or removing constant false alarm noise interference. Since the spectrum region of interest where the point cloud density needs to be increased additionally is determined by the projection of the real target in the environment, the reflection points in the region where the signal-to-noise ratio meets certain conditions are likely to be reflection signals from the target of interest. Based on this, in the embodiment of the present application, the reflection points where the energy of the reflection points in the first spectrum region is greater than the energy threshold and the energy between adjacent resolution units meets the ratio condition can be regarded as suspected point clouds, and these reflection points can be considered as reasonable output points of the detection stage. Among them, the reference density can be set according to actual needs or experience.

[0091] Exemplarily, the reflection points where the energy between adjacent resolution units meets the ratio condition are, for example, reflection points where the energy ratio between adjacent resolution units is less than a specific threshold (for example, less than 25 dB).

[0092] In some implementations, point cloud detection based on point cloud detection density includes: for a second spectral region where the detection density is not greater than the reference density, that is, a spectral region where there is no need to increase the point cloud density, local peak detection and / or constant false alarm detection strategy can be used to perform point cloud detection.

[0093] As described above, the embodiment of the present application configures appropriate point cloud detection strategies for different spectrum regions based on the results of spectrum division and classification, and outputs a point cloud density that meets user needs. Detection strategies such as shielding local peak detection and / or removing constant false alarm noise interference are implemented for the first spectrum region of interest to solve the problem of low-density point clouds caused by inappropriate application of peak detection strategies to surface targets such as vehicles. For the second spectrum region that has relatively low point cloud density requirements and is prone to false alarms, detection strategies such as local peak detection and / or constant false alarm detection are adopted to control the point cloud density and false alarm probability. Therefore, the embodiment of the present application can greatly suppress the false alarm probability while enhancing the point cloud density of the target of interest.

[0094] In some implementations, the point cloud detection method provided in the embodiments of the present application can be applied to a millimeter wave radar system with a fixed point cloud detection process and does not support a local configuration detection strategy. In these implementations, performing point cloud detection based on point cloud detection density includes: if the point cloud detection density of the spectrum region of interest is greater than the reference density, executing Figure 5 The detection sub-method shown; if the point cloud detection density of the spectrum area of ​​interest is not greater than the reference density, a second point cloud detection is performed using the millimeter wave radar system to obtain a point cloud detection result.

[0095] like Figure 5 As shown, the detection sub-method of the embodiment of the present application includes the following steps S51 to S54.

[0096] S51, using the millimeter wave radar system to perform a first point cloud detection to obtain a point cloud detection result.

[0097] S52, after the first point cloud detection is completed, based on the point cloud detection result of the first point cloud detection, a detection point located in the focus spectrum area is obtained as a focus point.

[0098] S53, obtaining surrounding resolution units of the focus point.

[0099] Exemplarily, when acquiring surrounding resolution units, it can be ensured that all acquired surrounding resolution units are located in the spectrum region of interest. The number and shape of the surrounding resolution units of the point of interest can be determined according to the target type to which the spectrum region belongs and the frequency range represented. For example, for a moving target in the near zone relative to the radar, since its Doppler frequency and azimuth span in the near zone are large, relatively more resolution units can be taken along the resolution unit where the point of interest is located in the Doppler and azimuth dimensions around it to cover as many effective target vehicle body reflection points as possible.

[0100] S54, adding the surrounding resolution units that meet the preset conditions into the output point cloud list of the first point cloud detection.

[0101] For example, for any resolution unit A in the surrounding resolution units, it can be determined in step S54 whether the energy of the resolution unit A is greater than a certain set threshold, and whether the ratio of the detection energy of the resolution unit A to the focus point and the energy ratio of the resolution unit A to the neighboring resolution units are within a certain threshold range. If the energy of the resolution unit A is greater than the set threshold, and the ratio of the detection energy to the focus point and the energy ratio to the neighboring resolution units are within the threshold range, it is determined that the resolution unit A meets the preset conditions and needs to be added to the output point cloud list of the first point cloud detection. Otherwise, it is determined that the resolution unit A does not meet the preset conditions and does not need to be added to the output point cloud list.

[0102] By traversing all the resolution units in the surrounding resolution units in the above manner, all the resolution units that need to be added to the output point cloud list are obtained. These resolution units are used as newly added point clouds and participate in subsequent operations together with the point clouds originally output by the first point cloud detection, such as horizontal angle estimation, pitch angle estimation, etc. For example, Figure 6 The final point cloud obtained by the above method is shown. Figure 6 As shown, in addition to the point cloud originally output by the first point cloud detection, the embodiment of the present application adds several point clouds that meet preset conditions as point clouds belonging to the target vehicle.

[0103] As described above, for a system that does not support a local configuration difference point cloud detection strategy, the embodiment of the present application provides a method of adding effective points based on the original process detection points, which greatly improves the feasibility of the implementation of the solution.

[0104] The protection scope of the point cloud detection method provided in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0105] The embodiment of the present application also provides a point cloud detection device, which can implement the point cloud detection method provided by the embodiment of the present application. However, the implementation device of the point cloud detection method provided by the embodiment of the present application includes but is not limited to the structure of the point cloud detection device listed in the embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0106] Figure 7 is a schematic block diagram of a point cloud detection device provided in an embodiment of the present application. Figure 7 As shown, the point cloud detection device 700 includes a spectrum division and classification module 701, a detection density determination module 702, and a point cloud detection module 703. The spectrum division and classification module 701 is used to divide the point cloud detection spectrum into regions to obtain spectrum regions and obtain the target type of each spectrum region. The detection density determination module 702 is used to determine the point cloud detection density of the spectrum region according to the target type. The point cloud detection module 703 is used to perform point cloud detection based on the point cloud detection density.

[0107] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0108] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, the point cloud detection method provided by the embodiment of the present application is implemented. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.

[0109] An embodiment of the present application also provides an electronic device. Figure 8 is a schematic block diagram of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 800 includes: at least one processor 801, a memory 802, at least one network interface 803 and a user interface 805. The various components in the electronic device 800 are coupled together through a bus system 804. It can be understood that the bus system 804 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 804 is not described in detail. Figure 8 In the specification, various buses are labeled as bus systems.

[0110] The user interface 805 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0111] It is understood that the memory 802 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present application is intended to include but is not limited to these and any other suitable categories of memory.

[0112] The memory 802 in the embodiment of the present application is used to store various categories of data to support the operation of the electronic device 800. Examples of these data include: any executable program for operating on the electronic device 800, such as an operating system 8021 and an application 8022; the operating system 8021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 8022 may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The point cloud detection method provided in the embodiment of the present application may be included in the application 8022.

[0113] The method disclosed in the above embodiment of the present application can be applied to the processor 801, or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 801 or an instruction in the form of software. The above processor 801 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 801 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor 801 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0114] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0115] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0116] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0121] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0122] In summary, the present application provides a point cloud detection method, device, storage medium and electronic device. The embodiment of the present application divides and classifies the point cloud detection spectrum into local areas, obtains multiple spectrum areas corresponding to different target types, and determines the point cloud detection density of each spectrum area. Point cloud detection is performed based on the determined point cloud detection density, and the focus target can be switched freely according to user needs, laying an important foundation for the system to output an ideal point cloud detection effect. Compared with the prior art, the embodiment of the present application can improve the integration of the point cloud detection stage with the actual scene and post-processing, and improve the flexibility of the millimeter wave radar system to correspond to multiple application scenarios. In addition, the embodiment of the present application has no special requirements for the front-end waveform design, signal processing flow, etc., and has high applicability.

[0123] The embodiment of the present application configures appropriate point cloud detection strategies for different spectrum regions based on the results of spectrum division and classification, and outputs a point cloud density that meets user needs. Detection strategies such as shielding local peak detection and / or removing constant false alarm noise interference are implemented for the first spectrum region of the target of interest to solve the problem of low-density point clouds caused by inappropriate application of peak detection strategies to surface targets such as vehicles. For the second spectrum region that has relatively low point cloud density requirements and is prone to false alarms, detection strategies such as local peak detection and / or constant false alarm detection are adopted to control the point cloud density and false alarm probability. Therefore, the embodiment of the present application can greatly suppress the false alarm probability while enhancing the point cloud density of the target of interest.

[0124] For systems that do not support local configuration difference point cloud detection strategies, the embodiments of the present application provide a method of adding new valid points by utilizing the original process detection points, which greatly improves the feasibility of implementing the solution.

[0125] Therefore, the present invention effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0126] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A point cloud detection method, characterized in that: The point cloud detection method comprises: Dividing the point cloud detection spectrum into regions to obtain spectrum regions, and obtaining the target type of each spectrum region; Determining a point cloud detection density of the spectrum region according to the target type; Point cloud detection is performed based on the point cloud detection density.

2. The point cloud detection method according to claim 1, characterized in that: The point cloud detection spectrum is divided into regions to obtain spectrum regions, and the target type of each spectrum region is obtained, including: Obtain post-processed target classification results; Obtaining target dimension information corresponding to each type of target in the target classification result, wherein the target dimension is an information dimension that can be represented by the point cloud detection spectrum; Projecting each type of target in the target classification result to the point cloud detection spectrum according to the target dimension information; The spectrum region and its target type are obtained according to the projection result.

3. The point cloud detection method according to claim 1, characterized in that: The point cloud detection spectrum is divided into regions to obtain spectrum regions, and the target type of each spectrum region is obtained, including: The point cloud detection spectrum is divided into regions using a spectrum division and classification model to obtain the spectrum regions and acquire the target type of each spectrum region, wherein the spectrum division and classification model is a trained deep learning model.

4. The point cloud detection method according to claim 1, characterized in that: The point cloud detection spectrum is divided into regions to obtain spectrum regions, and the target type of each spectrum region is obtained, including: Obtain the lateral distance of each reflection point in the reference point cloud data; Obtaining the point cloud proportion of each lateral distance by using a statistical method; Obtain corresponding target dimension information according to the lateral distance and radial distance of the stationary point, wherein the target dimension is an information dimension that can be represented by the point cloud detection spectrum, and the stationary point is a reflection point corresponding to the lateral distance whose point cloud proportion is greater than a ratio threshold; The frequency spectrum region corresponding to the stationary target and / or the moving target is acquired according to the target dimension information.

5. The point cloud detection method according to claim 1, characterized in that: Determining the point cloud detection density of the spectrum region according to the target type includes: determining the point cloud detection density of the spectrum region corresponding to the target type according to a detection accuracy requirement of the target type.

6. The point cloud detection method according to claim 1, characterized in that: Performing point cloud detection based on the point cloud detection density includes: For the first spectrum region where the detection density is greater than the reference density, point cloud detection is performed by shielding local peak detection and / or removing constant false alarm noise interference strategies, and reflection points in the first spectrum region where the energy of the reflection points is greater than the energy threshold and the energy between adjacent resolution units meets the ratio condition are taken as output points; For a second frequency spectrum region where the detection density is not greater than the reference density, a local peak detection and / or a constant false alarm detection strategy is used to perform point cloud detection.

7. The point cloud detection method according to claim 1, characterized in that: A millimeter wave radar system applied to a fixed point cloud detection process and not supporting a local configuration detection strategy, performing point cloud detection based on the point cloud detection density includes: If the point cloud detection density of the concerned spectral region is greater than the reference density, then: Performing a first point cloud detection using the millimeter wave radar system to obtain a point cloud detection result; Based on the point cloud detection result, acquiring a detection point located in the focus spectrum region as a focus point; Acquire surrounding resolution units of the focus point; Adding the surrounding resolution units that meet the preset conditions into the output point cloud list of the first point cloud detection; If the point cloud detection density of the concerned spectrum region is not greater than the reference density, then: The millimeter wave radar system is used to perform a second point cloud detection to obtain a point cloud detection result.

8. A point cloud detection device, characterized in that: The point cloud detection device comprises: A spectrum division and classification module is used to divide the point cloud detection spectrum into regions to obtain spectrum regions and obtain the target type of each spectrum region; A detection density determination module, used to determine the point cloud detection density of the spectrum region according to the target type; A point cloud detection module is used to perform point cloud detection based on the point cloud detection density.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud detection method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The electronic device comprises: A memory storing a computer program; A processor is communicatively connected to the memory, and executes the point cloud detection method according to any one of claims 1 to 7 when calling the computer program.

Images