Weak target detection and area division method and system based on millimeter-wave radar in oblique cabin

Abstract

The present invention discloses a method and system for weak target detection and area division based on millimeter-wave radar in an oblique cabin, including: performing Doppler-FFT transformation on the pre-processed target raw data to obtain a distance Doppler map, and then obtaining three-dimensional point cloud data; inputting the three-dimensional point cloud data into PointNet to obtain the classification of the three seats in the back row; based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three seats in the back row, obtaining the seat occupancy information of the living target in the back row. The present invention simplifies the data processing process and improves efficiency by using PointNet, and through feature fusion technology, the PointNet network can capture local and global information at the same time, enhancing the performance of the model. At the same time, the present invention makes a credibility judgment on this situation by comprehensively judging the state label output by the PointNet network and the radial distance of the strongest energy point; comprehensively judges the seat occupancy information according to the actual situation to reduce the misjudgment rate.

Description

Technical Field

[0001] The invention belongs to the technical field of neural networks and relates to a method and system for weak target detection and area division based on a millimeter-wave radar in an obliquely mounted cabin. Background Art

[0002] As cars become a standard part of the family, the number of car accidents is increasing, especially the serious consequences of children being left in cars in hot weather. The purpose of this technology is to avoid tragedies caused by adults neglecting to leave children or pets in the car through high-precision detection methods.

[0003] The in-cabin liveness detection technology aims to provide a solution that can sense objects in the vehicle after the vehicle is parked and locked. Millimeter-wave radar technology is one of the key means to achieve this goal. It can detect whether there is life in the vehicle by sensing the vital signs of people in the vehicle, such as breathing or limb movement. However, existing liveness detection technologies such as infrared or thermal imaging technology, cameras, etc. have some defects and cannot detect whether there is life in the vehicle in a timely and accurate manner. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art that the existing living body detection technology is easily affected by the environment, may cause privacy issues, etc., and cannot detect whether there are living bodies in the car in a timely and accurate manner, and to provide a method and system for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The method of weak target detection and area division based on millimeter-wave radar in the oblique cabin includes:

[0007] Obtain the target raw data collected by the radar and pre-process the target raw data;

[0008] Perform Doppler-FFT transformation on the pre-processed target raw data to obtain a range Doppler map;

[0009] Based on the range Doppler map, three-dimensional point cloud data is obtained;

[0010] Input the 3D point cloud data into PointNet to obtain the classification of the three rear seats;

[0011] Based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three rear seats, the rear-row living target seat occupancy information is obtained.

[0012] A further improvement of the present invention is:

[0013] Furthermore, the original data of the target collected by the radar is obtained, specifically: the electromagnetic wave data emitted by the radar and the echo data reflected back are mixed to obtain an intermediate frequency signal; the intermediate frequency signal is sampled to obtain the original data of the target.

[0014] Furthermore, the target raw data is preprocessed, specifically: data preprocessing is performed based on the data pre-whitening method to eliminate data redundancy; moving targets are displayed after using range-FFT, and static clutter is suppressed by using a single delay line canceller, that is, the echo of the static target is eliminated by subtracting two adjacent echo signals, and the echo of the moving target is extracted.

[0015] Further, the pre-processed target raw data is subjected to Doppler-FFT transformation to obtain a range Doppler map, specifically: the target raw data contains multiple pulses;

[0016] Perform FFT transformation on each pulse to obtain distance information, and then perform FFT transformation on the signal at each range unit to obtain Doppler information;

[0017] The distance processed data are organized into a matrix form, where each row represents a distance unit and each column represents a pulse;

[0018] Perform FFT transformation on each range unit to convert the time domain signal of the pulse into the Doppler domain;

[0019] The Doppler processed data matrix was input into the visualization tool as image data to obtain the range Doppler map.

[0020] Furthermore, based on the range Doppler map, three-dimensional point cloud data is obtained, specifically:

[0021] Perform CFAR target extraction based on the range Doppler map to obtain strong target points;

[0022] Based on each strong target point, array angle measurement processing is performed through multi-channel data to obtain the distance, speed, angle, and scattering intensity information of the point target;

[0023] The distance, speed, angle, and scattering intensity information of the point target are geometrically mapped, and the target information is output in the form of a point cloud.

[0024] Furthermore, the three-dimensional point cloud data is input into PointNet to obtain the classification of the three seats in the back row. Specifically, the input of PointNet is a set of all point cloud data of a frame, represented as an n*3 2d tensor, where n represents the number of point clouds and 3 represents the corresponding XYZ coordinates; first, the point cloud data is multiplied and aligned with a transformation matrix learned by T-Net; it is mapped to a 64-dimensional space through an MLP multi-layer perceptron, and then aligned using T-Net, and finally mapped to a 1024-dimensional space through two convolutions; for each point, there is a 1024-dimensional vector representation; a pooling operation is performed to obtain a 1024-dimensional global variable; the global feature is output by the MLP multi-layer perceptron. The size of the 3 classification value scores determines the classification of the input point cloud; finally, the category with the highest score is selected as the prediction result.

[0025] Furthermore, based on the radial distance of the maximum energy value in the 3D point cloud data and the classification of the three seats in the back row, the seat occupancy information of the living target in the back row is obtained. Specifically, the radial distance of the top three energy values ​​in the point cloud data is calculated. Take the mean as the center of the weak target micro-motion; if the classification result is the mean of the radial distances of the top three energy values ​​in the point cloud data If the mean division result is consistent, the position of the weak target in the back row is output; if the classification result is consistent with the radial distance mean of the top three energy values If the judgment results are inconsistent, the farthest radial distance is calculated based on the middle position of the left, middle and right seats and classified. The farthest radial distance from the center of the corresponding seat is selected based on the classification result. ; The radial distance average of the top three energy values ​​in the point cloud data and the maximum radial distance Assign different weights and sum them up; make a second judgment on the area range to get the final area occupied by the weak target.

[0026] Furthermore, the average radial distance between the classification result and the top three energy values ​​in the point cloud data If the mean division results are consistent, the position of the weak target in the rear row is output. Specifically, the length of the rear seat is divided into three equal parts; the installation height of the millimeter-wave radar in the cabin is , and then the farthest radial distance R that the millimeter-wave radar beam can illuminate to the rear area is obtained as follows: , , ; The back row is divided into the areas according to the farthest radial distance R: Back row left: 1- , — , — -1; if the classification result is the average of the radial distances to the top three energy values ​​in the point cloud data If the mean division results are consistent, the location of the weak target in the back row is output.

[0027] Furthermore, if the classification result is the mean of the radial distances between the top three energy values If the judgment results are inconsistent, the farthest radial distance is calculated based on the middle position of the left, middle and right seats and classified. Specifically, the farthest radial distance is calculated based on the middle position of the left, middle and right seats. , , Select the corresponding seat center's farthest radial distance based on the classification result The average radial distance of the top three energy values ​​in the point cloud data and the maximum radial distance Assign different weights and sum them up; make a second judgment on the area range to get the final area occupied by the weak target.

[0028] The weak target detection and area division system based on the millimeter-wave radar in the oblique cabin includes:

[0029] A preprocessing module, wherein the preprocessing module obtains the target raw data collected by the radar and preprocesses the target raw data;

[0030] A Fourier transform module, wherein the Fourier transform module performs Doppler-FFT transform on the pre-processed target raw data to obtain a range Doppler map;

[0031] A first acquisition module, wherein the first acquisition module acquires three-dimensional point cloud data based on a range Doppler map;

[0032] A second acquisition module, wherein the second acquisition module inputs the three-dimensional point cloud data into PointNet to obtain the classification of the three rear seats;

[0033] The third acquisition module acquires the seat occupancy information of the living target in the rear row based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three rear seats.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention performs Doppler-FFT transformation on the pre-processed target raw data to obtain a distance Doppler map, and then obtains three-dimensional point cloud data; the three-dimensional point cloud data is input into PointNet to obtain the classification of the three seats in the back row; based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three seats in the back row, the seat occupancy information of the living target in the back row is obtained. By using PointNet, the present invention simplifies the data processing process and improves efficiency, and through feature fusion technology, PointNet can capture local and global information at the same time to enhance the performance of the model. At the same time, the present invention makes a comprehensive judgment on the state label output by the PointNet network and the radial distance of the strongest energy point, and uses the radial distance of the strongest energy point to make a credibility judgment on this situation. Not only based on the classification output result, the seat occupancy information is comprehensively judged according to the actual situation to reduce the misjudgment rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 It is a flow chart of the method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin of the present invention;

[0038] Figure 2 It is a structural schematic diagram of the system for weak target detection and area division based on the millimeter-wave radar in the oblique cabin of the present invention;

[0039] Figure 3 This is a schematic diagram of the network framework structure of PointNet;

[0040] Figure 4 Schematic diagram of the maximum radial distance R of the obliquely mounted millimeter-wave radar;

[0041] Figure 5 Schematic diagram of data processing and algorithm structure. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0043] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0045] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0046] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0047] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0049] See also Figure 1 The present invention discloses a method for weak target detection and area division based on millimeter-wave radar in an oblique cabin, comprising:

[0050] S101, obtaining target raw data collected by the radar, and preprocessing the target raw data;

[0051] The original target data collected by the radar is obtained by mixing the electromagnetic wave data emitted by the radar and the echo data reflected back to obtain an intermediate frequency signal; the intermediate frequency signal is sampled to obtain the original target data.

[0052] The target raw data is preprocessed, specifically: data preprocessing is performed based on the data pre-whitening method to eliminate data redundancy; moving targets are displayed after using range-FFT, and static clutter is suppressed by using a single delay line canceller, that is, the echo of the static target is eliminated by subtracting two adjacent echo signals, and the echo of the moving target is extracted.

[0053] S102, performing Doppler-FFT transformation on the pre-processed target raw data to obtain a range Doppler map;

[0054] The target raw data contains multiple pulses; each pulse is subjected to FFT transformation to obtain the distance information, and then the signal at each distance unit is subjected to FFT transformation to obtain the Doppler information;

[0055] The distance processed data are organized into a matrix form, where each row represents a distance unit and each column represents a pulse;

[0056] Perform FFT transformation on each range unit to convert the time domain signal of the pulse into the Doppler domain;

[0057] The Doppler processed data matrix was input into the visualization tool as image data to obtain the range Doppler map.

[0058] S103, acquiring three-dimensional point cloud data based on the range Doppler map;

[0059] S103.1, perform CFAR target extraction based on the range Doppler map to obtain strong target points;

[0060] CFAR is an algorithm for radar signal processing that can achieve adaptive detection of targets in the presence of large background clutter. The CFAR algorithm effectively detects target signals by setting a constant false alarm rate and dynamically adjusting the detection threshold based on the statistical characteristics of the background clutter.

[0061] In CFAR target extraction based on range-Doppler maps, the following steps are usually followed:

[0062] The range Doppler image is preprocessed, including denoising and filtering, to improve image quality.

[0063] The pre-processed range-Doppler map is subjected to CFAR processing. This usually involves selecting training cells, protection cells, and test cells. The training cells are used to estimate the statistical characteristics of background clutter, the protection cells are used to prevent the target signal from leaking into the training cells, and the test cells are the target areas to be detected.

[0064] Training cells: A certain number of training cells are selected, which should be located around the test cell and far away from the target signal to estimate the statistical characteristics of the background clutter.

[0065] Guard cells: A certain number of guard cells are set around the test cell to prevent the target signal from leaking into the training cell. The number and location of the guard cells should be selected according to the actual situation.

[0066] Test unit: The target area to be tested, that is, the unit that needs to be processed by CFAR.

[0067] Threshold calculation: Calculate the detection threshold based on the statistical characteristics of the training unit. This threshold will be used to determine whether the target signal exists in the test unit.

[0068] Target detection: The signal strength in the test cell is compared with the calculated threshold. If the signal strength exceeds the threshold, the target is determined to exist and is extracted as a strong target point.

[0069] S103.2, based on each strong target point, array angle measurement processing is performed through multi-channel data to obtain the distance, speed, angle, and scattering intensity information of the point target;

[0070] The distance of the point target is determined by the radar by measuring the time difference between the transmitted pulse and the received echo.

[0071] The speed of the point target is that when there is relative motion between the target and the radar, the frequency of the received echo signal will shift, and the shift is proportional to the target's speed. The target's speed can be calculated by measuring the frequency shift;

[0072] ;

[0073] in, is the Doppler shift, v is the target's velocity relative to the radar, and λ is the wavelength of the radar's transmitted signal.

[0074] The angle of the point target is: multiple receiving channels collect the echo signal of the target at the same time, and use the phase difference between the echo signals received by multiple antennas to measure the angle. There is a nonlinear relationship between the phase difference and the target angle, and the angle information of the target can be indirectly solved by calculating the phase difference.

[0075] When the signal transmitted by the radar encounters the target and is reflected back, antennas at different locations receive echo signals with different phases. These phase differences are caused by the different path lengths that the signal travels when reaching different antennas.

[0076] If there are two antennas, the distance between them is d, the wavelength of the radar signal is λ, and the incident angle of the target relative to the antenna array is θ. Then, the phase difference between the echo signals received by the two antennas is It can be expressed as:

[0077] ;

[0078] where d⋅sin(θ) is the difference in path length between the signal reaching the two antennas.

[0079] The scattering intensity information is the echo signal of the target collected by the radar system through multiple receiving channels at the same time. These channels are usually distributed at different positions of the radar antenna to form an array. The collected multi-channel data is sent to the array signal processing unit, and the array signal processing unit uses the phase difference and amplitude difference between the signals received by the array antenna to perform angle measurement processing on the target. Based on the array angle measurement processing, the radar system evaluates the scattering intensity of the target by measuring the amplitude of the target echo. Among them, the scattering intensity is usually proportional to the power of the target echo, so the scattering intensity information is obtained by measuring the power of the echo signal.

[0080] S103.3, geometrically map the distance, speed, angle, and scattering intensity information of the point target, and output the target information in the form of a point cloud.

[0081] Establish a three-dimensional coordinate system to represent the spatial position of point targets. Formulate rules for mapping point targets from real space to the three-dimensional coordinate system based on distance, speed, and angle information. Convert distance information to coordinate values, convert speed information to visual attributes such as color or size, and determine the direction of the point based on angle information.

[0082] Scattering intensity mapping: Map the scattering intensity information to the intensity value or color depth of the point cloud to reflect the surface characteristics of the target object.

[0083] Point cloud data generation: Based on the results of geometric mapping, point cloud data containing point target position, velocity, angle and scattering intensity information is generated.

[0084] Point cloud format selection: Select an appropriate point cloud format to store the generated point cloud data.

[0085] Point cloud output: Output the generated point cloud data to a file for visualization.

[0086] S104, inputting the three-dimensional point cloud data into PointNet to obtain the classification of the three rear seats;

[0087] The input of PointNet is a collection of all point cloud data of a frame, represented as an n*3 2d tensor, where n represents the number of point clouds and 3 represents the corresponding XYZ coordinates. First, the point cloud data is multiplied and aligned with a transformation matrix learned by T-Net. It is mapped to a 64-dimensional space through an MLP multilayer perceptron, aligned once using T-Net, and finally mapped to a 1024-dimensional space through two convolutions. For each point, there is a 1024-dimensional vector representation. A pooling operation is then performed to obtain a 1024-dimensional global variable. The global feature is output by the MLP multilayer perceptron to determine the classification of the input point cloud based on the size of the 3 classification value scores, and finally the category with the highest score is selected as the prediction result.

[0088] Suppose the scores output by MLP are: back row left: 0.7, back row middle: 0.2, back row right: 0.1; Determine the classification: The size of the classification score determines which category the point cloud belongs to; the back row left has the highest score, so it is predicted that there is someone in the back row left represented by this point cloud.

[0089] The PointNet network finally outputs the score of each category, i.e. 0.7 for the back left, 0.2 for the back center, and 0.1 for the back right, and selects the category with the highest score as the predicted classification result.

[0090] S105, based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three rear seats, obtain the rear row living target seat occupancy information.

[0091] The radial distance of the top three energy values ​​in the point cloud data Take the mean as the center of the weak target micro-motion; if the classification result is the mean of the radial distances of the top three energy values ​​in the point cloud data If the mean division result is consistent, the position of the weak target in the back row is output; if the classification result is consistent with the radial distance mean of the top three energy values If the judgment results are inconsistent, the farthest radial distance is calculated based on the middle position of the left, middle and right seats and classified. The farthest radial distance from the center of the corresponding seat is selected based on the classification result. ; The radial distance average of the top three energy values ​​in the point cloud data and the maximum radial distance Assign different weights and sum them up; make a second judgment on the area range to get the final area occupied by the weak target.

[0092] This classification result is the final area division result of PointNet output; that is, the back row left or back row right or back row center, and the radial distance of the top three point clouds with the highest energy value is calculated to get the average value. .

[0093] The average radial distance between the classification result and the top three energy values ​​in the point cloud data If the mean division results are consistent, the position of the weak target in the rear row is output. Specifically, the length of the rear seat is divided into three equal parts; the installation height of the millimeter-wave radar in the cabin is , and then the farthest radial distance R that the millimeter-wave radar beam can illuminate to the rear area is obtained as follows: , , ; The back row is divided into the areas according to the farthest radial distance R: Back row left: 1- , Back row: — , Back row right: — -1; if the classification result is the average of the radial distances to the top three energy values ​​in the point cloud data If the mean division results are consistent, the position of the weak target in the back row is output;

[0094] In interval 1— In the equation, 1 means that the distance cannot be 0, so the calculation starts from a value greater than 0.

[0095] If the classification result is the average of the radial distances to the top three energy values If the judgment results are inconsistent, the farthest radial distance is calculated based on the middle position of the left, middle and right seats and classified. Specifically, the farthest radial distance is calculated based on the middle position of the left, middle and right seats. , , Select the corresponding seat center's farthest radial distance based on the classification result The average radial distance of the top three energy values ​​in the point cloud data and the maximum radial distance Assign different weights and sum them up; make a second judgment on the area range to get the final area occupied by the weak target;

[0096] in, , , It represents the center position of the three regional results output by PointNet, and then uses the radial distance mean of the three strongest points for credibility weighting processing, combining the network output position with the three points with the strongest energy for judgment.

[0097] Since the output of the PointNet network is not 100% accurate, when the classification result is The areas where they fall are not consistent, and the location where the average of the three points with the strongest energy is located is more credible. and Perform credibility weighting processing to obtain a more credible final result.

[0098] See also Figure 2 The present invention discloses a system for weak target detection and area division based on a millimeter-wave radar in an obliquely mounted cabin, comprising:

[0099] A preprocessing module, wherein the preprocessing module obtains the target raw data collected by the radar and preprocesses the target raw data;

[0100] A Fourier transform module, wherein the Fourier transform module performs Doppler-FFT transform on the pre-processed target raw data to obtain a range Doppler map;

[0101] A first acquisition module, wherein the first acquisition module acquires three-dimensional point cloud data based on a range Doppler map;

[0102] A second acquisition module, wherein the second acquisition module inputs the three-dimensional point cloud data into PointNet to obtain the classification of the three rear seats;

[0103] The third acquisition module acquires the seat occupancy information of the living target in the rear row based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three rear seats.

[0104] The present invention discloses a method for weak target detection and area division based on a millimeter-wave radar in an obliquely mounted cabin, comprising:

[0105] Step 1, radar original signal acquisition and processing: In a detection cycle, the millimeter wave radar transmits electromagnetic waves through the transmitting antenna, and the receiving antenna receives the reflected echo data. The mixer combines the transmitting signal and the receiving signal to generate an intermediate frequency signal. The ADC module samples the intermediate frequency signal to obtain the original ADC data;

[0106] Step 2, static clutter filtering: Data pre-whitening method is used for data pre-processing to eliminate data redundancy. Moving target indicator is performed after using range-FFT, and static clutter is suppressed by using a single delay line canceller, that is, the echo of the static target is eliminated by subtracting two adjacent echo signals, and the echo of the moving target is extracted;

[0107] Step 3, point cloud output: Doppler-FFT is performed to obtain the range-Doppler map (RDM), and CFAR targets are performed on the RDM to extract strong target points. Then, array angle measurement is performed through multi-channel data to obtain information such as the distance, speed, angle, and scattering intensity of the point target. Finally, the distance and angle information of the point target are geometrically mapped and the target information can be output in the form of a point cloud.

[0108] Step 4, PointNet classification: The above point cloud is input into PointNet for training. PointNet is a deep learning network specifically used to process point cloud data. The input of PointNet is a set of all point cloud data of a frame, represented as an n*3 2d tensor, where n represents the number of point clouds and 3 represents the corresponding XYZ coordinates. First, the data is aligned by multiplying the transformation matrix learned by T-Net to ensure the invariance of the model for specific spatial transformations. It is mapped to a 64-dimensional space through MLP, then aligned using T-Net, and finally mapped to a 1024-dimensional space through two convolutions. At this time, for each point, there is a 1024-dimensional vector representation. Then a pooling operation is performed to obtain a 1024-dimensional global variable. The global features can be passed through MLP to output the probability of each class.

[0109] The global features extracted by PointNet can accomplish the classification task well. The framework structure of PointNet is as follows Figure 3 As shown. The first T-Net 3×3 is used to align the input point cloud data. The second T-Net 64×64 is used to align the 64-dimensional features. The MLP multi-layer perceptron is used to extract the features of the point cloud, and convolution with shared weights is used here. Maxpooling is used to summarize the information of all point clouds, perform maximum pooling, and obtain the global features of the point cloud. Using the features, the characteristics of the point cloud can be learned based on this feature to achieve the classification task. Finally, the classification of the input point cloud can be determined based on the size of the output 3 classification value scores.

[0110] After inputting the point cloud into the PointNet trained for weak targets, the classification of the three seats in the back row will be obtained. However, the weak target may not be located in the middle of the seat, which will cause the classification value scores to be close and lead to misjudgment. Therefore, it is necessary to increase the credibility condition to assist the region division result. Using the X and Y in the point cloud data to calculate the radial distance R to increase the credibility:

[0111] ;

[0112] The radial distance between the top three energy values ​​in the point cloud data The average value is taken as the center of the weak target micro-motion and is considered to be the location of the weak target. The boundary value of the rear seats is calculated by the longest radial distance to divide the area. Figure 4 shown.

[0113] Criteria for dividing the rear seats: Assume that the length of the rear seats is , divided into three equal parts: , , The millimeter wave radar is installed at a height of The maximum radial distances R that the millimeter-wave radar beam can illuminate to the rear area are: , , Therefore, the rear row is divided into the areas with the longest radial distance R as follows: Rear row left: 1- , — , — -1. If the classification result is the average of the radial distances of the top three energy values ​​in the point cloud data If the mean division result is consistent, the position of the weak target in the back row is output. If the judgment results are inconsistent, the farthest radial distance is calculated from the middle position of the left, middle and right seats. , , . Select the corresponding seat center's farthest radial distance based on the classification result The average radial distance of the top three energy values ​​in the point cloud data Add 70% confidence and 30% confidence to the classification result. = 0.7* +0.3* . Then it falls into the area range for secondary judgment to obtain the final area occupied by the weak target. The overall process of the algorithm is as follows Figure 5 shown.

[0114] When PointNet results in someone in the back row on the left, It shows that there is someone in the back row; the result is subjected to secondary analysis. The calculation is based on the left middle of the back row output by PointNet, that is, is the center point, For the base, calculate the hypotenuse length as the radial distance from the back row to the left. Figure 4 The length of the first dashed line in Then calculate =0.7* +0.3* Get the final result.

[0115] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for weak target detection and area division based on millimeter-wave radar in an oblique cabin, characterized in that: include: Obtain the target raw data collected by the radar and pre-process the target raw data; Perform Doppler-FFT transformation on the pre-processed target raw data to obtain a range Doppler map; Based on the range Doppler map, three-dimensional point cloud data is obtained; Input the 3D point cloud data into PointNet to obtain the classification of the three rear seats; The input of PointNet is a collection of all point cloud data of a frame, represented as an n*3 2d tensor, where n represents the number of point clouds and 3 represents the corresponding XYZ coordinates. First, the point cloud data is multiplied and aligned with a transformation matrix learned by T-Net. It is mapped to a 64-dimensional space through an MLP multilayer perceptron, and then aligned using T-Net, and finally mapped to a 1024-dimensional space through two convolutions. For each point, there is a 1024-dimensional vector representation. A pooling operation is then performed to obtain a 1024-dimensional global variable. The global features are output by the MLP multilayer perceptron to determine the classification of the input point cloud based on the size of the 3 classification value scores. Finally, the category with the highest score is selected as the prediction result. Based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three rear seats, the rear-row living target seat occupancy information is obtained.

2. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 1 is characterized in that: The acquisition of the target original data collected by the radar specifically includes: mixing the electromagnetic wave data emitted by the radar and the echo data reflected back to obtain an intermediate frequency signal; and sampling the intermediate frequency signal to obtain the target original data.

3. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 2 is characterized in that: The target raw data is preprocessed, specifically: data preprocessing is performed based on a data pre-whitening method to eliminate data redundancy; moving target display is performed after using range-FFT, and static clutter is suppressed by using a single delay line canceller, that is, the echo of the static target is eliminated by subtracting two adjacent echo signals, and the echo of the moving target is extracted.

4. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 3 is characterized in that: The pre-processed target raw data is subjected to Doppler-FFT transformation to obtain a range Doppler map, specifically: the target raw data includes a plurality of pulses; Perform FFT transformation on each pulse to obtain distance information, and then perform FFT transformation on the signal at each range unit to obtain Doppler information; The distance processed data are organized into a matrix form, where each row represents a distance unit and each column represents a pulse; Perform FFT transformation on each range unit to convert the time domain signal of the pulse into the Doppler domain; The Doppler processed data matrix was input into the visualization tool as image data to obtain the range Doppler map.

5. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 4 is characterized in that: The three-dimensional point cloud data is obtained based on the range Doppler map, specifically: Perform CFAR target extraction based on the range Doppler map to obtain strong target points; Based on each strong target point, array angle measurement processing is performed through multi-channel data to obtain the distance, speed, angle, and scattering intensity information of the point target; The distance, speed, angle, and scattering intensity information of the point target are geometrically mapped, and the target information is output in the form of a point cloud.

6. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 5 is characterized in that: The method of obtaining the seat occupancy information of the living target in the back row based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three seats in the back row is specifically as follows: the radial distance of the top three highest energy values ​​in the point cloud data is calculated. Take the mean as the micro-motion center of the weak target; If the classification result is the average of the radial distances between the top three energy values ​​in the point cloud data If the mean division result is consistent, the position of the weak target in the back row is output; if the classification result is consistent with the radial distance mean of the top three energy values If the judgment results are inconsistent, the farthest radial distance is calculated based on the middle position of the left, middle and right seats and classified. The farthest radial distance from the center of the corresponding seat is selected based on the classification result. ; The radial distance average of the top three energy values ​​in the point cloud data and the maximum radial distance Assign different weights and sum them up; make a second judgment on the area range to get the final area occupied by the weak target.

7. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 6 is characterized in that: The average radial distance between the classification result and the top three energy values ​​in the point cloud data If the mean division results are consistent, the position of the weak target in the rear row is output. Specifically, the length of the rear seat is divided into three equal parts; the installation height of the millimeter-wave radar in the cabin is , and then the farthest radial distance R that the millimeter-wave radar beam can illuminate to the rear area is obtained as follows: , , ; The rear row is divided into the areas with the longest radial distance R: 1- , — , — -1; if the classification result is the average of the radial distances to the top three energy values ​​in the point cloud data If the mean division results are consistent, the location of the weak target in the back row is output.

8. The method for weak target detection and area division based on millimeter-wave radar in an obliquely mounted cabin according to claim 7 is characterized in that: If the classification result is the average of the radial distances between the top three energy values If the judgment results are inconsistent, the farthest radial distance is calculated based on the middle position of the left, middle and right seats and classified. Specifically, the farthest radial distance is calculated based on the middle position of the left, middle and right seats. , , Select the corresponding seat center's farthest radial distance based on the classification result The average radial distance of the top three energy values ​​in the point cloud data and the maximum radial distance Assign different weights and sum them up; make a second judgment on the area range to get the final area occupied by the weak target.

9. Based on the millimeter-wave radar weak target detection and area division system in the oblique cabin, it is characterized by: include: A preprocessing module, wherein the preprocessing module obtains the target raw data collected by the radar and preprocesses the target raw data; A Fourier transform module, wherein the Fourier transform module performs Doppler-FFT transform on the pre-processed target raw data to obtain a range Doppler map; A first acquisition module, wherein the first acquisition module acquires three-dimensional point cloud data based on a range Doppler map; A second acquisition module, wherein the second acquisition module inputs the three-dimensional point cloud data into PointNet to obtain the classification of the three rear seats; The third acquisition module acquires the seat occupancy information of the living target in the rear row based on the radial distance of the maximum energy value in the three-dimensional point cloud data and the classification of the three rear seats.

Images