Self-learning method, system and vehicle based on millimeter-wave radar target fusion boundary

Abstract

The present invention relates to the field of autonomous driving technology, and more particularly to a self-learning method, system, and vehicle based on millimeter-wave radar target fusion boundaries. The method comprises collecting waveform signal data of a target obstacle, extracting scattered point information of the target obstacle based on the waveform signal data, denoising the scattered point information to obtain point cloud data information of the target obstacle, fitting the point cloud data information to obtain a target point cloud fusion boundary, subjecting the target fusion boundary to measured target data for calculation, subjecting boundary point cloud data that meets the requirements to an algorithm to calculate a radar output target value, determining the type of target obstacle for boundary fusion, and iteratively optimizing a target obstacle fusion boundary database. In this way, target obstacle detection and recognition loss is avoided, and the target fusion boundary is adjusted and optimized to be close to the actual size of the target object, thereby achieving more refined target fusion and improving the accuracy of target obstacle recognition.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a self-learning method, system and vehicle based on millimeter-wave radar target fusion boundaries. Background Art

[0002] Nowadays, cars are increasingly becoming a convenient means of transportation. With the rise in car ownership, people are increasingly concerned about their comfort and safety. Autonomous driving technology, an essential component of intelligent driving, is attracting widespread attention. However, a key aspect of autonomous driving relies on the perception of the surrounding environment through multiple sensors. Among these sensors, millimeter-wave radar holds a significant position due to its penetrating properties, robustness to inclement weather, and immunity to light. Automotive millimeter-wave radar transmits millimeter waves to a target, receives the reflected millimeter-wave signal, and uses the Doppler effect and the speed of light to determine the target's distance, speed, direction, and angle of motion.

[0003] There are existing target recognition methods based on millimeter-wave radar in related technologies. The main method is to use FFT (Fast Fourier Transform) to parse the original target data of the millimeter-wave radar and then perform target fusion on the original point cloud data. The data is processed and classified by collecting target featureless points. However, the target boundary of the original point cloud target data of the millimeter-wave radar is fixed, and it is not possible to achieve self-learning optimization of the fusion boundary of the point cloud target data. At the same time, the point cloud fusion algorithm is prone to identify two similar and relatively close targets, or two targets with one strong and one weak as one target, resulting in target loss. Summary of the Invention

[0004] The technical solution of this application aims to solve the problem of the fixed boundary of the millimeter-wave radar original point cloud target data, so that the target fusion boundary has self-learning ability to achieve more powerful target fusion capability and reduce the occurrence of target loss. Therefore, this application proposes a self-learning method, system and vehicle based on the millimeter-wave radar target fusion boundary.

[0005] In a first aspect, an embodiment of the present application provides a self-learning method based on millimeter-wave radar target fusion boundaries, comprising:

[0006] Building a target obstacle fusion boundary database, wherein the target obstacle fusion boundary database includes original point cloud data information of preset different target obstacles;

[0007] Continuously collect waveform signal data of the target obstacle, extract characteristic value information of the target obstacle based on the waveform signal data, determine the scattered point information of the target obstacle based on the characteristic value information, and denoise the scattered point information by combining physical denoising and clustering algorithm, wherein the scattered point information includes the spatial information, distance information, velocity information and reflection peak value of each scattered point;

[0008] The point cloud data of the target obstacle is determined based on the denoised scattered point information. After weighted averaging and Bayesian statistics, the Lagrange interpolation method is used to fit the target point cloud fusion boundary. The target fusion boundary is then brought into the measured target data for calculation.

[0009] According to the output result of target fusion boundary calculation, determine whether the target fusion boundary satisfies the original point cloud data;

[0010] If the requirements are met, the boundary point cloud data that meets the requirements is brought into the algorithm to calculate the radar output target value, determine the type of target obstacle, and iteratively optimize the target obstacle fusion boundary database.

[0011] According to some embodiments of the present application, the target obstacle fusion boundary database is built, and the target obstacle fusion boundary database includes preset original point cloud data information of different target obstacles, including:

[0012] The target obstacles are classified according to their structural characteristics, and the original point cloud data information of each type of target obstacle is preset and stored, wherein the original point cloud data information includes the number of point clouds of each target obstacle, the point cloud coverage area, the point cloud contour distribution information and the signal reflection intensity.

[0013] According to some embodiments of the present application, determining the point cloud data information of the target obstacle based on the denoised scattered point information, performing weighted averaging and Bayesian statistics on the point cloud data information, and then performing Lagrange interpolation to fit the target point cloud fusion boundary, and bringing the target fusion boundary into the measured target data for calculation, includes:

[0014] A weight N is assigned to each point cloud data sample by weighted averaging, where the larger the radar signal strength RCS value, the larger the weight N value.

[0015] Use the Bayesian statistical boundary to determine the probability R of whether the point cloud is included. When the probability R>R1, it is confirmed that the boundary contains the point cloud and the corresponding data is updated. The original value is V, and the new boundary value V1=V+KV1, where K is the boundary coefficient.

[0016] All the new boundary values ​​obtained are obtained through Lagrange interpolation method to obtain the boundary formula The point cloud data is fitted with a boundary curve to obtain a fitting curve. The middle value of the curve is selected based on the comparison between the fitting curve and the boundary formula to obtain a new boundary curve.

[0017] According to some embodiments of the present application, all the new boundary values ​​are obtained by Lagrange interpolation to obtain the boundary formula The point cloud data is fitted with a boundary curve to obtain a fitting curve, and the middle value of the curve is selected based on the comparison between the fitting curve and the boundary formula to obtain a new boundary curve. It also includes: establishing a radar point cloud coordinate system, determining the horizontal coordinate, vertical coordinate, horizontal speed, longitudinal speed, radar reflection area RCS, relative distance and status value of the boundary point cloud based on the point cloud data information, and extracting the relative position of each point cloud data point.

[0018] According to some embodiments of the present application, the target fusion boundary is calculated and outputted, and the determination of whether the target fusion boundary satisfies the original point cloud data includes: matching the target fusion boundary with the preset target obstacle point cloud quantity, point cloud coverage area, point cloud contour distribution information and signal reflection intensity one by one according to the target fusion boundary calculation and outputted result, and determining whether the target fusion boundary satisfies the original point cloud data according to the matching result.

[0019] According to some embodiments of the present application, if the requirements are met, the boundary point cloud data that meets the requirements is brought into the algorithm to calculate the radar output target value, determine the type of target obstacle, and iteratively optimize the target obstacle fusion boundary database, including: according to the radar output target value, using a convolutional neural network model to iteratively extract the boundary feature values ​​of different target obstacles from the boundary data, and using multiple convolution kernels to detect different boundary feature values.

[0020] According to some embodiments of the present application, the method of iteratively extracting boundary feature values ​​of different target obstacles using a convolutional neural network model based on the target value output by the radar, and detecting different boundary feature values ​​using multiple convolution kernels, includes:

[0021] The convolution operation is completed based on the sliding window calculation. Each window calculates the boundary feature value of a local area. The pooling layer is used for downsampling. By performing aggregating operations on part of the boundary data, the dimension and calculation amount of the data are reduced. Maximum pooling and average pooling are used to retain the main features of the input data. The fully connected layer connects the boundary output of the pooling layer to the final output layer for classification, regression and other tasks. Each node of the fully connected layer is connected to all nodes in the previous layer to realize self-learning iterative processing of boundary data.

[0022] In a second aspect, an embodiment of the present application provides a self-learning system based on millimeter-wave radar target fusion boundaries, the system comprising:

[0023] a data acquisition module configured to continuously collect target obstacle information around the target vehicle through the millimeter-wave radar, wherein the target obstacle information includes scattered point information and spatial information, distance information, speed information, and reflection peak value of each scattered point;

[0024] The data processing module is configured to determine the point cloud data information of the target obstacle based on the denoised scattered point information, perform weighted averaging and Bayesian statistics on the point cloud data information, and then use Lagrange interpolation to fit the target point cloud fusion boundary, and then bring the target fusion boundary into the measured target data for calculation;

[0025] The boundary fusion module is configured to calculate the output result based on the target fusion boundary, determine whether the target fusion boundary satisfies the original point cloud data, and perform fusion processing on the boundary of the target obstacle based on the original point cloud data;

[0026] The feature extraction module is configured to use a convolutional neural network model to iteratively extract boundary feature values ​​of different target obstacles based on the target value output by the radar, and use multiple convolution kernels to detect different boundary feature values;

[0027] The iterative processing module is configured to perform self-learning iterative processing on the target fusion boundary of the original point cloud data in the target obstacle fusion boundary database according to the boundary feature value obtained by the feature extraction module and the collected point cloud data information of the target obstacle.

[0028] In a third aspect, an embodiment of the present application further provides a vehicle, comprising:

[0029] processor;

[0030] a memory for storing instructions executable by the processor;

[0031] Wherein, the processor is configured to:

[0032] Implement the steps of the self-learning method based on millimeter-wave radar target fusion boundary described in the embodiment of the first aspect above;

[0033] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of a self-learning method based on millimeter-wave radar target fusion boundary described in the embodiment of the first aspect above.

[0034] Compared with the prior art, the technical solution provided by the embodiments of the present application includes at least the following beneficial effects:

[0035] Millimeter-wave radar continuously collects waveform signal data of target obstacles. Based on the waveform signal data, scattered point information of the target obstacles is extracted. The scattered point information is denoised using a combination of physical denoising and clustering algorithms to obtain point cloud data of the target obstacles. The obtained point cloud data is then weighted averaged and Bayesian statistics are performed, followed by Lagrange interpolation to fit the target point cloud fusion boundary. The target fusion boundary is then incorporated into the measured target data for calculation. The boundary point cloud data that meets the requirements is then incorporated into the algorithm to calculate the radar output target value, determine the type of target obstacle, and perform boundary fusion based on a preset target obstacle fusion boundary database. Simultaneously, the target obstacle fusion boundary database is iteratively optimized. In this way, the target obstacle boundary recognition can be enhanced through the target fusion boundary, avoiding target obstacle detection and recognition loss. At the same time, by performing self-learning optimization and iteration on the fusion boundary of the original point cloud target data in the preset target obstacle fusion boundary database, the target fusion boundary is continuously adjusted and optimized to approach the actual size of the target object, achieving more refined target fusion and improving the accuracy of target obstacle recognition.

[0036] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 is a flow chart of a self-learning method based on millimeter-wave radar target fusion boundary according to an embodiment of the present application;

[0039] Figure 2 is another flow chart of a self-learning method based on millimeter-wave radar target fusion boundary according to an embodiment of the present application;

[0040] Figure 3 This is a block diagram of a self-learning system based on millimeter-wave radar target fusion boundaries according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present application are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0043] Only portions relevant to the present application are shown in the accompanying drawings, not all of them. Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0044] As used in this specification, the terms "component," "module," "system," "unit," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or distributed between two or more computers. In addition, these units can be executed from various computer-readable media having various data structures stored thereon. Units can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from a second unit interacting with another unit in a local system, a distributed system, and / or a network, such as the Internet, which interacts with other systems via signals).

[0045] Example 1

[0046] See also Figure 1 This embodiment provides a self-learning method based on millimeter-wave radar target fusion boundary, including:

[0047] Step S100: Building a target obstacle fusion boundary database, wherein the target obstacle fusion boundary database includes original point cloud data information of different preset target obstacles;

[0048] In this step, it should be noted that the method steps of this embodiment can be used for vehicle autonomous driving. The vehicle-mounted millimeter-wave radar transmits millimeter waves to the target object, receives the millimeter-wave signal reflected by the target object, and obtains the target's distance, speed information, movement direction, angle and other information based on the Doppler effect and the speed of light propagation to determine the information of the target obstacle.

[0049] It is understandable that target obstacles can have different types of attributes and can be in motion or stationary state. Of course, the physical objects corresponding to the target obstacles can be large buses, cars, two-wheeled vehicles, pedestrians, road signs, manhole covers, street lights, guardrails and other targets. The target obstacle fusion boundary database includes preset raw point cloud data information of different target obstacles. The raw point cloud data information of different target obstacles can be classified according to the structural characteristics of the target obstacles, and the raw point cloud data information of each type of target obstacle is preset and stored. The raw point cloud data information includes the number of point clouds of each target obstacle, the point cloud coverage area, the point cloud contour distribution information and the signal reflection intensity.

[0050] It is also understood that when the millimeter-wave radar collects obstacle information, the original point cloud data information of each type of target obstacle preset in the target obstacle fusion boundary database can be retrieved by the processor, and can also be preset in the target obstacle fusion boundary database for iterative optimization. At the same time, the initial data stored in the target obstacle fusion boundary database can be preset through experience.

[0051] What needs to be explained is that point cloud data is actually a method used to represent the external obstacle information collected by millimeter wave radar. The obstacle information is represented in the virtual space through point cloud data, and different target obstacles are identified through point cloud fusion algorithm. Specifically, the number of point clouds of each target obstacle, the point cloud coverage area, the point cloud contour distribution information and the signal reflection intensity are analyzed and calculated.

[0052] Thus, the status of the target obstacle can be monitored at any time;

[0053] Step S200: Continuously collecting waveform signal data of the target obstacle, extracting characteristic value information of the target obstacle based on the waveform signal data, determining scattered point information of the target obstacle based on the characteristic value information, and denoising the scattered point information using a combination of physical denoising and a clustering algorithm, wherein the scattered point information includes spatial information, distance information, velocity information, and reflection peak value of each scattered point;

[0054] In this step, it should be noted that the vehicle-mounted millimeter-wave radar transmits millimeter waves to detect target obstacles. When a target obstacle exists within the detection range of the millimeter-wave radar, the millimeter-wave radar transmits millimeter waves, which are reflected. The millimeter-wave radar receives echo information returned by the millimeter-wave radar, divides the echo information into periods, obtains all target original messages detected in each period, and performs data analysis on the target original messages to obtain characteristic value information of the target obstacle. It is understood that the characteristic value information of the target obstacle includes the true value of a series of information such as the number of target obstacles detected in the close-range beam scanning of the current detection period, the target's position information, the target's speed information, the target's reflection intensity RCS (Radar Cross Section) area value, and the target ID number.

[0055] It is understood that when a target obstacle exists within the detection range of the millimeter-wave radar, subsequent data processing and display are often performed by outputting a point cloud, and the scattered point information of the target obstacle is determined by the characteristic value information. It is understood that the scattered point information includes the motion state and position information of each scattered point. For example, the scattered point information includes the spatial information, distance information, speed information, and reflection peak value of each scattered point.

[0056] Since there is a lot of noise in the obtained scattered point information, in order to optimize subsequent data processing and improve the accuracy of detection, it is often necessary to use a combination of physical denoising and clustering algorithms to denoise the scattered point information. For example, the clustering algorithm processing includes: starting from a moving point that has not been assigned to the target cluster and has not been visited, calculating the neighborhood whose coordinates and speed differences with the moving point are within a certain threshold. In this embodiment, the threshold for the coordinate difference is set to 0.15m, and the threshold for the speed difference is set to 1.5m / s, that is, other moving points with a plane distance of no more than 0.2m from the point and an absolute value of the speed difference of no more than 1.5m / s are all within the neighborhood of the point. If the total number of points falling within the neighborhood exceeds the point threshold (for example, 50), the current moving point is marked as the first point in the new cluster, otherwise, the moving point is marked as a noise point; the marked moving point status is recorded as visited. It should be understood that the point threshold can be set according to different scenarios;

[0057] The first point in the new cluster is processed, and all points in its neighborhood are marked as the new cluster. The neighborhood of all moving points in the new cluster is processed. After all the neighborhoods of all moving points are processed and no new moving points are added to the new cluster, the above processing is completed in the current cluster, and a new unvisited point is extracted and processed, so that the next cluster or noise will be found. This process is repeated until all points are marked as visited. After the processing is completed, each moving point is marked as belonging to a cluster or a noise point. The sum of the signal-to-noise ratios of the points in each cluster is calculated, and the cluster with a sum of signal-to-noise ratios greater than a threshold (for example, 150dB) is marked as the target cluster, otherwise it is marked as a noise cluster. It should be understood that the threshold of the sum of the signal-to-noise ratios can be set according to different scenarios.

[0058] Step S300: Determine the point cloud data information of the target obstacle based on the denoised scattered point information, perform weighted averaging and Bayesian statistics on the point cloud data information, and then use Lagrange interpolation to fit the target point cloud fusion boundary. The target fusion boundary is then brought into the measured target data for calculation.

[0059] In this step, the point cloud data information of the target obstacle is determined based on the scattered point information after denoising. The point cloud data information is obtained by denoising the scattered point information. In the subsequent processing process, it is necessary to perform weighted averaging and Bayesian statistics on the point cloud data information and then use Lagrange interpolation method to fit the target point cloud fusion boundary. The target fusion boundary is then brought into the measured target data for calculation;

[0060] Specifically, a weight N is given to each point cloud data sample by weighted averaging, wherein the larger the radar signal strength RCS value, the larger the weight N value;

[0061] Use the Bayesian statistical boundary to determine the probability R of whether the point cloud is included. When the probability R>R1, it is confirmed that the boundary contains the point cloud and the corresponding data is updated. The original value is V, and the new boundary value V1=V+KV1, where K is the boundary coefficient.

[0062] All the new boundary values ​​obtained are obtained through Lagrange interpolation method to obtain the boundary formula , where yᵢ is the ordinate parameter of the new boundary value, The horizontal coordinate parameter of the new boundary value is used, and the point cloud data is fitted with a boundary curve to obtain a fitting curve. The middle value of the curve is selected by comparing the fitting curve with the boundary formula to obtain a new boundary curve. Of course, before the data is processed, it also includes establishing a radar point cloud coordinate system, determining the horizontal coordinate, vertical coordinate, horizontal speed, longitudinal speed, radar reflection area RCS, relative distance and status value of the boundary point cloud based on the point cloud data information, and extracting the relative position of each point cloud data point.

[0063] It should be noted that the purpose of this step is to find the target boundary of the detected actual target obstacle to represent the virtual state of the target obstacle and improve the recognition of the target obstacle.

[0064] Step S400: judging whether the target fusion boundary satisfies the original point cloud data according to the target fusion boundary calculation output result;

[0065] In this step, based on the target fusion boundary calculation output result, it is determined whether the target fusion boundary satisfies the original point cloud data, including: matching the target fusion boundary with the preset target obstacle point cloud quantity, point cloud coverage area, point cloud contour distribution information and signal reflection intensity one by one according to the target fusion boundary calculation output result, and determining whether the target fusion boundary satisfies the original point cloud data based on the matching result;

[0066] In some embodiments, it should be noted that if the target fusion boundary does not match the original point cloud data, it means that the target obstacle is not stored in the target obstacle fusion boundary database. The processor will then store the detected target obstacle point cloud data and number it for subsequent target fusion boundary matching with the target obstacle point cloud quantity, point cloud coverage area, point cloud contour distribution information and signal reflection intensity one by one, thereby improving the accuracy of target obstacle detection and matching.

[0067] Step S500: If the requirements are met, the boundary point cloud data that meets the requirements is brought into the algorithm to calculate the radar output target value, determine the type of target obstacle, and iteratively optimize the target obstacle fusion boundary database.

[0068] In this step, it should be noted that meeting the requirements means that the detected target obstacle successfully matches the original point cloud data. Then, by retrieving the preset point cloud data information of the target obstacle, the point cloud data of the actual detected target obstacle is updated using the preset original point cloud data, and the target boundary is fused. In this way, the fixed boundary of the cloud target data is solved, and the self-learning optimization processing of the fusion boundary of the point cloud target data is achieved. For two similar and relatively close targets, or two targets with one strong and one weak, it is easy to identify them as one target, thus avoiding loss.

[0069] In some embodiments, an iterative optimization process is performed on the target obstacle fusion boundary database, including using a convolutional neural network model to iteratively extract boundary feature values ​​of different target obstacles based on the radar output target value, using multiple convolution kernels to detect different boundary feature values, and completing the convolution operation based on the sliding window calculation. The boundary feature value of a local area is calculated for each window. The pooling layer is used for downsampling. By performing aggregating operations on partial areas of the boundary data, the dimension and computational complexity of the data are reduced. Maximum pooling and average pooling are used to retain the main features in the input data. The fully connected layer connects the boundary output of the pooling layer to the final output layer for classification, regression and other tasks. Each node of the fully connected layer is connected to all nodes of the previous layer to realize self-learning iterative processing of the boundary data.

[0070] Example 2

[0071] See also Figure 2 This embodiment is a practical application process of one of the above-mentioned self-learning methods based on millimeter-wave radar target fusion boundaries, including:

[0072] Step S1, detecting multiple groups of targets based on millimeter wave radar, detecting the same target, multiple targets, and different locations;

[0073] Step S2, filtering and processing the point cloud data according to the multiple sets of target obstacle point cloud data obtained in S1;

[0074] Step S3, classifying the target points according to the target obstacle point cloud data in S2 based on the RCS value, target point speed and other information;

[0075] In this step, exemplarily, the preset target obstacles include large vehicles, cars, two-wheel vehicles, pedestrians, and other types of target obstacles; the target point classification includes extracting the internal target distance information A, target speed information B, target angle information C, target intensity signal D, the number of original target point clouds E drawn by the initial boundary, and the coverage range F of the original target point clouds from the waveform signal received by the radar. When the radar receives the target point cloud data, it is judged according to the target signal intensity D value that when D > X, it may be a vehicle target Q1, when D is between X and Y values, it may be a two-wheel car target Q2, and when D < Y value, it is a pedestrian or other target Q3. From the S1 target, its speed B, the number of original target point clouds E, and the coverage range F of the original point clouds are extracted. If E > the quantity limit value E1 and F > the range limit value F1, the target is a large vehicle; if E > E1 and F < F1, the target is a car; if E < E1 and F > F1, the target is a large vehicle. From the Q3 target, the speed B, the number of original target point clouds E, and the coverage range F of the original point clouds are extracted. If B > B3, E > E3, and F > F3, the target is other targets; if B < B3, E < E3, and F < F3, the target is a pedestrian; if B > B3, E < E3, and F < F3, the target is other moving targets; if B > B3, E > E3, and F < F3, the target is other targets; if B < B3, E > E3, and F < F3, the target is a pedestrian; if B < B3, E < E3, and F > F3, the target is other targets.

[0076] Step S4, check whether the current target is a large vehicle. If it is a large vehicle, proceed to step S5; if not, proceed to step S6;

[0077] Step S5, store the target data in the large vehicle database for subsequent processing;

[0078] Step S6, compare the target data in S3 to check whether it is a car. If it is a car, proceed to step S7; if not, proceed to step S8;

[0079] Step S7, store the target data in the car database for subsequent processing;

[0080] Step S8, compare the target data in S3 to check whether it is a two-wheel vehicle. If it is a two-wheel vehicle, proceed to step S9; if not, proceed to step S10;

[0081] Step S9, store the target data in the two-wheel vehicle database for subsequent processing;

[0082] Step S10, compare the target data in S3 to check whether it is a pedestrian. If it is a pedestrian, proceed to step S11; if not, proceed to step S12;

[0083] Step S11, store the target data in the pedestrian database for subsequent processing;

[0084] The remaining targets in step S12 and S3 are other types of targets, including road signs, manhole covers, street lights, guardrails and other targets;

[0085] Step S13, storing the target data in other databases for subsequent processing;

[0086] In step S14, the data in S5, S7, S9, S11, and S13 are weighted averaged and Bayesian statistics are performed, and then the Lagrange interpolation method is used to fit the target point cloud fusion boundary;

[0087] Step S15, bringing the target fusion boundary into the measured target data for calculation;

[0088] Step S16, checking whether the target fusion boundary satisfies the point cloud data according to the target output point cloud data of S15, if so, proceed to step S17, if not, proceed to steps S2 to S16;

[0089] Step S17: The boundary data that meets the requirements is brought into the algorithm to calculate the radar output target value, and whether the target value is correct is determined. If the target value is correct, the millimeter-wave radar target fusion boundary self-learning iterative processing is completed. If it does not meet the requirements, steps S14 to S17 are performed to redesign the target fusion boundary.

[0090] In this step, the self-learning iterative processing and redesign of the target fusion boundary include using a convolutional neural network model to iteratively extract boundary features of different targets from boundary data. The convolution layer extracts features from boundary data through convolution operations, and multiple convolution kernels can be used to detect different features. The convolution operation is performed using a sliding window calculation, and each window calculates the features of a local area. The pooling layer is used for downsampling, reducing the dimension and computational complexity of the data by performing a pooling operation on part of the boundary data. Maximum pooling and average pooling are used to retain the main features in the input data. The fully connected layer connects the boundary output of the pooling layer to the final output layer for classification, regression, and other tasks. Each node in the fully connected layer is connected to all nodes in the previous layer to achieve iterative self-learning of boundary data.

[0091] Example 3

[0092] See also Figure 3 This embodiment provides a self-learning system based on millimeter-wave radar target fusion boundaries. The self-learning system 200 based on millimeter-wave radar target fusion boundaries includes:

[0093] The data acquisition module 210 is configured to continuously collect target obstacle information around the target vehicle through the millimeter wave radar, wherein the target obstacle information includes each scattered point information and the spatial information, distance information, speed information and reflection peak value of each scattered point;

[0094] The data processing module 220 is configured to determine the point cloud data information of the target obstacle based on the denoised scattered point information, perform weighted averaging and Bayesian statistics on the point cloud data information, and then use Lagrange interpolation to fit the target point cloud fusion boundary, and then bring the target fusion boundary into the measured target data for calculation;

[0095] The boundary fusion module 230 is configured to determine whether the target fusion boundary satisfies the original point cloud data based on the target fusion boundary calculation output result, and perform fusion processing on the boundary of the target obstacle based on the original point cloud data;

[0096] The feature extraction module 240 is configured to use a convolutional neural network model to iteratively extract boundary feature values ​​of different target obstacles based on the target value output by the radar, and use multiple convolution kernels to detect different boundary feature values;

[0097] The iterative processing module 250 is configured to perform self-learning iterative processing on the target fusion boundary of the original point cloud data in the target obstacle fusion boundary database according to the boundary feature value obtained by the feature extraction module 240 and the collected point cloud data information of the target obstacle.

[0098] Example 4

[0099] The present application also provides a vehicle, including:

[0100] processor;

[0101] a memory for storing instructions executable by the processor;

[0102] Wherein, the processor is configured to:

[0103] The steps of implementing the self-learning method based on millimeter-wave radar target fusion boundary described in the above embodiment.

[0104] Example 5

[0105] An embodiment of the present application also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of a self-learning method based on millimeter-wave radar target fusion boundary described in the above embodiment are implemented.

[0106] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0111] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0112] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves. It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0113] In the specification, claims, and accompanying drawings of this application, the terms "first," "second," "third," and the like are used to distinguish different objects and are not used to describe a particular order. Furthermore, the terms "including," "comprising," "having," and any variations thereof are intended to cover non-exclusive inclusions. For example, a list of steps or elements may be included, or alternatively, steps or elements not listed may be included, or other steps or elements may be included that are inherent to the process, method, product, or apparatus.

[0114] Only portions relevant to the present application are shown in the accompanying drawings, not all of them. Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0115] As used in this specification, the terms "component," "module," "system," "unit," and the like are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or distributed between two or more computers. In addition, these units can be executed from various computer-readable media having various data structures stored thereon. Units can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from a second unit interacting with another unit in a local system, a distributed system, and / or a network, such as the Internet, which interacts with other systems via signals).

[0116] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "example," "specific example," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0117] Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present embodiment application. The appearance of this phrase in various positions in the specification does not necessarily mean that they are all the same embodiments, nor are they independent or alternative embodiments that are mutually exclusive with other embodiments. It can be understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of this application.

[0118] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A self-learning method based on millimeter-wave radar target fusion boundary, characterized in that: include: Establishing a target obstacle fusion boundary database, the target obstacle fusion boundary database includes preset raw point cloud data information of different target obstacles; wherein, the target obstacles are classified according to their structural characteristics, and the raw point cloud data information of each type of target obstacle is preset and stored, wherein the raw point cloud data information includes the number of point clouds of each target obstacle, the point cloud coverage area, the point cloud contour distribution information, and the signal reflection strength; Continuously collect waveform signal data of target obstacles, extract characteristic value information of the target obstacles based on the waveform signal data, determine the scattered point information of the target obstacles based on the characteristic value information, and denoise the scattered point information by combining physical denoising and clustering algorithms. The scattered point information includes spatial information, distance information, velocity information, and reflection peak value of each scattered point; The point cloud data of the target obstacle is determined based on the denoised scattered point information. The point cloud data is weighted averaged and Bayesian statistics are performed, and then the Lagrange interpolation method is used to fit the target point cloud fusion boundary. The target fusion boundary is then brought into the measured target data for calculation. This includes establishing a radar point cloud coordinate system, determining the horizontal and vertical coordinates, horizontal and vertical speeds, radar reflection area (RCS), relative distance, and status values ​​of the boundary point cloud based on the point cloud data information, and extracting the relative position of each point cloud data point. Based on the target fusion boundary calculation output result, determine whether the target fusion boundary satisfies the original point cloud data; this includes matching the target fusion boundary with the preset target obstacle point cloud quantity, point cloud coverage area, point cloud contour distribution information, and signal reflection intensity one by one based on the target fusion boundary calculation output result, and determining whether the target fusion boundary satisfies the original point cloud data based on the matching result; If the requirements are met, the boundary point cloud data that meets the requirements is brought into the algorithm to calculate the radar output target value, determine the type of target obstacle, and iteratively optimize the target obstacle fusion boundary database.

2. The self-learning method based on millimeter wave radar target fusion boundary according to claim 1 is characterized in that: The method of determining the point cloud data information of the target obstacle based on the denoised scattered point information, performing weighted averaging and Bayesian statistics on the point cloud data information, and then fitting the target point cloud fusion boundary using the Lagrange interpolation method, and bringing the target fusion boundary into the measured target data for calculation includes: A weight N is assigned to each point cloud data sample by weighted averaging, where the larger the radar signal strength RCS value, the larger the weight N value. Use the Bayesian statistical boundary to determine the probability R of whether the point cloud is included. When the probability R>R1, it is confirmed that the boundary contains the point cloud and the corresponding data is updated. The original value is V, and the new boundary value V1=V+KV1, where K is the boundary coefficient. All the new boundary values ​​obtained are obtained through Lagrange interpolation method to obtain the boundary formula The point cloud data is fitted with a boundary curve to obtain a fitting curve. The middle value of the curve is selected based on the comparison between the fitting curve and the boundary formula to obtain a new boundary curve.

3. The self-learning method based on millimeter wave radar target fusion boundary according to claim 1, characterized in that: If the requirements are met, the boundary point cloud data that meets the requirements is brought into the algorithm to calculate the radar output target value, determine the type of target obstacle, and iteratively optimize the target obstacle fusion boundary database, including: based on the radar output target value, using a convolutional neural network model to iteratively extract boundary feature values ​​of different target obstacles from the boundary data, and using multiple convolution kernels to detect different boundary feature values.

4. The self-learning method based on millimeter wave radar target fusion boundary according to claim 3 is characterized in that: According to the target value output by the radar, a convolutional neural network model is used to iteratively extract boundary feature values ​​of different target obstacles using boundary data, and multiple convolution kernels are used to detect different boundary feature values, including: The convolution operation is completed based on the sliding window calculation. Each window calculates the boundary feature value of a local area. The pooling layer is used for downsampling. By performing aggregating operations on part of the boundary data, the dimension and calculation amount of the data are reduced. Maximum pooling and average pooling are used to retain the main features of the input data. The fully connected layer connects the boundary output of the pooling layer to the final output layer for classification, regression and other tasks. Each node of the fully connected layer is connected to all nodes in the previous layer to realize self-learning iterative processing of boundary data.

5. A system corresponding to the self-learning method based on millimeter-wave radar target fusion boundary according to any one of claims 1 to 4, characterized in that: The system comprises: a data acquisition module configured to continuously collect target obstacle information around the target vehicle through the millimeter-wave radar, wherein the target obstacle information includes scattered point information and spatial information, distance information, speed information, and reflection peak value of each scattered point; The data processing module is configured to determine the point cloud data information of the target obstacle based on the denoised scattered point information, perform weighted averaging and Bayesian statistics on the point cloud data information, and then use Lagrange interpolation to fit the target point cloud fusion boundary, and then bring the target fusion boundary into the measured target data for calculation; The boundary fusion module is configured to calculate the output result based on the target fusion boundary, determine whether the target fusion boundary satisfies the original point cloud data, and perform fusion processing on the boundary of the target obstacle based on the original point cloud data; The feature extraction module is configured to use a convolutional neural network model to iteratively extract boundary feature values ​​of different target obstacles based on the target value output by the radar, and use multiple convolution kernels to detect different boundary feature values; The iterative processing module is configured to perform self-learning iterative processing on the target fusion boundary of the original point cloud data in the target obstacle fusion boundary database according to the boundary feature value obtained by the feature extraction module and the collected point cloud data information of the target obstacle.

6. A vehicle, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to: Implement the steps of a self-learning method based on millimeter-wave radar target fusion boundary as described in any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of a self-learning method based on millimeter-wave radar target fusion boundary according to any one of claims 1 to 4 are implemented.

Images