A millimeter wave radar point cloud generation method based on a sliding window and related products

Abstract

The application discloses a millimeter wave radar point cloud generation method based on a sliding window and related products. The method comprises the following steps: acquiring received data of a millimeter wave radar; performing distance fast Fourier transform along a fast time dimension and performing static background removal to obtain distance slow time data; extracting a plurality of fixed length sub-sequences along a slow time dimension by using a sliding time window and performing Doppler fast Fourier transform to generate a plurality of data blocks; aggregating distance Doppler spectra generated by the plurality of data blocks and performing constant false alarm rate detection to identify distance Doppler velocity units containing targets; extracting data in an antenna dimension as snapshots and performing MUSIC super-resolution angle estimation on the plurality of snapshots to obtain angle information; and combining the distance information and the angle information to generate a millimeter wave radar point cloud. Through the scheme, a target reflection point can be fully analyzed in combination with multiple dimensions, and a point cloud with higher density can be generated.

Description

Technical Field

[0001] This application generally relates to the field of millimeter-wave radar technology. More specifically, this application relates to a method, system, computer-readable storage medium, and electronic device for generating millimeter-wave radar point clouds based on a sliding window. Background Technology

[0002] Millimeter-wave radar can sense targets in the environment (such as the human body) by transmitting and receiving millimeter-wave signals. Point cloud data can describe the reflection positions of different parts of the human body in space, and is an important foundation for downstream sensing applications of millimeter-wave radar. In existing technologies, point cloud data is derived from the received radar data. However, existing point cloud generation technologies cannot fully utilize the received data from multiple dimensions, resulting in the underutilization of the multi-dimensional resolution capabilities of millimeter-wave radar, and limiting the density and attitude representation capabilities of the generated point cloud.

[0003] In view of this, there is an urgent need to provide a millimeter-wave radar point cloud generation scheme based on sliding windows, so as to fully analyze the target reflection points from multiple dimensions and thus generate a higher density point cloud. Summary of the Invention

[0004] In order to at least solve one or more of the technical problems mentioned above, this application proposes a method for generating millimeter-wave radar point clouds based on sliding windows and related products in several aspects.

[0005] In a first aspect, this application provides a method for generating millimeter-wave radar point clouds based on a sliding window, comprising: acquiring received data from a millimeter-wave radar, wherein the received data has a fast time dimension, a slow time dimension, and an antenna dimension; performing a range fast Fourier transform along the fast time dimension, and performing static background removal on the transformed data to obtain range slow time data; extracting multiple fixed-length subsequences along the slow time dimension using a sliding time window, and performing a Doppler fast Fourier transform within each fixed-length subsequence to generate multiple data blocks containing range, Doppler velocity, and antenna dimension; and generating multiple data blocks... The range-Doppler spectra are aggregated, and constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectra to identify range-Doppler velocity cells containing targets. For each detected range-Doppler velocity cell, its range information is recorded. For each detected range-Doppler velocity cell, antenna dimension data is extracted from the data block corresponding to each window as a snapshot, and super-resolution angle estimation based on multiple signal classification (MUSIC) is performed on multiple snapshots to obtain the angle information of the target reflection point. Finally, coordinate transformation is performed by combining the range information and the angle information to generate a millimeter-wave radar point cloud in Cartesian coordinates.

[0006] In some embodiments, the fast time dimension is converted to a distance dimension by the distance fast Fourier transform, such that the space is divided into multiple distance units, the received data includes dynamic target reflection signals, and the static background removal includes: calculating the average value along the slow time dimension for each distance unit; subtracting the average value from the signal of the corresponding distance unit to highlight the dynamic target reflection signals in the distance slow time data.

[0007] In some embodiments, the step of extracting multiple fixed-length subsequences along the slow time dimension using a sliding time window includes: using a pulse sequence in the slow time dimension as the sliding object; using a fixed-length window to slide backward from the starting position of the pulse sequence; extracting a fixed-length subsequence each time the window is slid; and ensuring that the same Doppler velocity unit in different windows corresponds to the same physical Doppler velocity.

[0008] In some embodiments, the length of the fixed-length window is determined by the number of antennas and the number of targets to be resolved.

[0009] In some embodiments, the aggregation of the range-Doppler spectra generated from multiple data blocks includes: averaging each data block containing range, Doppler velocity, and antenna dimension along the antenna dimension to obtain the range-Doppler spectrum of the corresponding window; and aggregating the range-Doppler spectra corresponding to different windows to form an aggregated range-Doppler spectrum.

[0010] In some embodiments, the constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectrum and is used to determine a range index and a Doppler velocity index containing the target, wherein the range index is used to obtain the distance to the target reflection point and the Doppler velocity index is used to locate antenna dimension data at the same physical Doppler velocity from data blocks corresponding to multiple windows.

[0011] In some embodiments, performing super-resolution angle estimation based on multiple signal classification (MUSIC) for multiple snapshots includes: performing autocorrelation processing on each snapshot to obtain the covariance matrix corresponding to that snapshot; averaging the covariance matrices corresponding to multiple snapshots to obtain the received signal covariance matrix for angle estimation; performing eigenvalue decomposition on the received signal covariance matrix to obtain the noise subspace; and scanning the angle spectrum based on the noise subspace and the steering vector to obtain the angle information of the target reflection point.

[0012] In some embodiments, the millimeter-wave radar point cloud is generated by coordinate transformation of the distance, azimuth, and elevation angle of the target reflection point.

[0013] In a second aspect, this application provides a millimeter-wave radar point cloud generation system based on a sliding window, comprising: a signal preprocessing module for acquiring received data from a millimeter-wave radar, performing a range fast Fourier transform along the fast time dimension, and performing static background removal on the transformed data to obtain range slow time data, wherein the received data has the fast time dimension, the slow time dimension, and the antenna dimension; a multi-window range-Doppler spectrum generation module for extracting multiple fixed-length subsequences along the slow time dimension using a sliding time window, and performing a Doppler fast Fourier transform within each fixed-length subsequence to generate multiple data blocks containing range, Doppler velocity, and antenna dimension; a target detection module for aggregating the range-Doppler spectra generated from the multiple data blocks, and performing constant false alarm rate detection on the aggregated range-Doppler spectrum to identify range-Doppler velocity units containing targets, wherein for each detected range-Doppler velocity unit, its range information is recorded; and a multi-snapshot construction module for extracting antenna dimension data as a snapshot from the data block corresponding to each window for each detected range-Doppler velocity unit. The MUSIC super-resolution angle estimation module is used to perform super-resolution angle estimation based on multiple signal classification MUSIC for multiple snapshots to obtain the angle information of the target reflection point; the point cloud generation module is used to combine the distance information and the angle information to perform coordinate transformation to generate a millimeter-wave radar point cloud in Cartesian coordinate system.

[0014] In a third aspect, this application provides an electronic device, comprising: a memory storing computer instructions for generating millimeter-wave radar point clouds based on a sliding window; and a processor executing the computer instructions, causing the electronic device to perform the millimeter-wave radar point cloud generation method based on a sliding window as described in the preceding and following embodiments.

[0015] Through the sliding window-based millimeter-wave radar point cloud generation method and related products provided above, this application embodiment supports Doppler velocity estimation and MUSIC super-resolution angle estimation simultaneously in the slow time dimension using a sliding window strategy. This enables millimeter-wave radar to jointly utilize three dimensions—range, Doppler velocity, and super-resolution angle—to resolve target reflection points, thereby generating a higher-density point cloud. Furthermore, by employing a sliding window on the slow-time pulse sequence, the complete pulse sequence is divided into multiple overlapping subsequences with temporal order. A Doppler Fast Fourier Transform is performed within each subsequence, thus preserving cross-window temporal diversity while obtaining Doppler velocity resolution. This solves the problem that the complete Doppler transform destroys time-series information, leading to ineffective decorrelation of super-resolution angle estimation. Attached Figure Description

[0016] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0017] Figure 1 A schematic block diagram of a computer-implemented millimeter-wave radar point cloud generation system based on a sliding window, according to an embodiment of this application, is shown. Figure 2 A flowchart illustrating a method for generating millimeter-wave radar point clouds based on a sliding window, according to an embodiment of this application, is shown. Figure 3 A flowchart illustrating another embodiment of the millimeter-wave radar point cloud generation method based on a sliding window is shown. Figure 4 A schematic block diagram illustrating an embodiment of this application with a sliding window and multiple snapshots is shown; and Figure 5 A flowchart illustrating the super-resolution angle estimation process of a two-dimensional antenna array according to an embodiment of this application is shown. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0021] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0022] Exemplary application scenarios In related technologies, radar received data typically includes fast time dimensions, slow time dimensions, and antenna dimensions. The fast time dimension is formed by sampling a single pulse and contains target distance information; the slow time dimension is formed by multiple continuously transmitted pulses and contains target Doppler velocity information; the antenna dimension is formed by the antenna array and contains target angle information. For macroscopic motion perception tasks such as human activity, posture recognition, fall detection, and indoor tracking, point cloud data can describe the reflection positions of different parts of the human body in space, serving as a crucial foundation for downstream perception applications of millimeter-wave radar.

[0023] Millimeter-wave radar can distinguish target reflection points in three dimensions: range, Doppler velocity, and angle. Range resolution is determined by bandwidth, Doppler velocity resolution by frame duration, and angular resolution by the number of antennas in the antenna array. Commercial millimeter-wave radars typically have only 3 to 8 antennas in a single direction, limiting their angular resolution and resulting in sparse point clouds when directly using Fast Fourier Transform or Digital Beamforming. To improve angular resolution, related technologies employ Multiple Signal Classification (MUSIC) algorithms and their variants for super-resolution angle estimation. The MUSIC algorithm constructs the received signal covariance matrix and performs eigenvalue decomposition to separate the signal subspace from the noise subspace. It then uses the orthogonality between the steering vector and the noise subspace to scan the angle spectrum, thereby obtaining the super-resolution angle estimation result.

[0024] The relevant millimeter-wave radar point cloud generation processes are mainly divided into two categories. The first category is based on range and Doppler velocity. It first performs a range fast Fourier transform on the fast time dimension, and then performs a Doppler fast Fourier transform on the slow time dimension. Target cells are detected on the range-Doppler spectrum, and angle estimation is performed using the antenna data corresponding to the target cells. This type of process can utilize target Doppler velocity information, but it only retains a single antenna array snapshot in a detected range-Doppler velocity cell, which is difficult to meet the MUSIC super-resolution angle estimation requirement of multiple snapshots for decoherence. Therefore, angle estimation is usually only performed using angle fast Fourier transform or digital beamforming. The second category is based on range and super-resolution angle. It first performs a range fast Fourier transform on the fast time dimension to detect target range cells on the range spectrum, and then extracts the slow time dimension and antenna dimension data of the corresponding range cell and feeds them into the MUSIC algorithm. Since the slow time series is still retained, signal coherence can be suppressed through multiple pulse snapshots, and super-resolution angle estimation can be achieved.

[0025] The inventors discovered in their research that neither of the aforementioned processes simultaneously utilizes the three dimensions of range, Doppler velocity, and super-resolution angle. The fundamental reason lies in the conflict between Doppler velocity estimation and super-resolution angle estimation regarding their use of the slow time dimension. Doppler velocity estimation requires performing a Fast Fourier Transform (FFT) along the slow time dimension to convert the time-domain pulse sequence into a Doppler spectrum. After this transformation, the frequency domain aggregation result corresponding to different timestamp signals for each Doppler velocity unit is lost, destroying the original time-series information. Super-resolution angle estimation, on the other hand, requires retaining multiple time snapshots. By averaging these snapshots along the time dimension, the coherence between signals from different reflection points is suppressed, making the rank of the covariance matrix close to the number of targets. If a full pulse sequence Doppler FFT is performed first, each velocity unit only provides a single snapshot, which cannot effectively perform decoherence processing, thus limiting the effectiveness of the MUSIC algorithm. Therefore, the aforementioned processes typically only combine range with Doppler velocity, or range with super-resolution angle, resulting in the millimeter-wave radar's resolution capabilities in all three dimensions not being fully utilized, thus limiting the generated point cloud density and attitude characterization capabilities.

[0026] To address the problems in the aforementioned scenarios, the inventors further discovered that a sliding window strategy can simultaneously support Doppler velocity estimation and MUSIC super-resolution angle estimation in the slow time dimension. This enables millimeter-wave radar to jointly utilize the three dimensions of range, Doppler velocity, and super-resolution angle to analyze target reflection points, thereby generating a higher density point cloud.

[0027] The following combination Figures 1-5 The scheme of this application is described in detail.

[0028] Figure 1A flowchart of a sliding window-based millimeter-wave radar point cloud generation system 100 according to an embodiment of this application is shown.

[0029] like Figure 1 As shown, the millimeter-wave radar point cloud generation system 100 based on sliding window may include a signal preprocessing module 101, a multi-window range Doppler spectrum generation module 102, a target detection module 103, a multi-snapshot construction module 104, a MUSIC super-resolution angle estimation module 105, and a point cloud generation module 106.

[0030] In some embodiments, the millimeter-wave radar point cloud generation system 100 based on sliding window (hereinafter referred to as the system 100) can be used in conjunction with millimeter-wave radar, that is, the system 100 itself may or may not include millimeter-wave radar.

[0031] The millimeter-wave radar is used for data acquisition and reception. The specific configurations of each module in system 100 are as follows: The signal preprocessing module 101 is used to acquire the received data of the millimeter-wave radar, perform a range fast Fourier transform along the fast time dimension, and perform static background removal on the transformed data to obtain range slow time data, wherein the received data has the fast time dimension, the slow time dimension and the antenna dimension.

[0032] The multi-window range-Doppler spectrum generation module 102 is used to extract multiple fixed-length subsequences along the slow time dimension using a sliding time window, and to perform Doppler fast Fourier transform within each fixed-length subsequence to generate multiple data blocks containing range, Doppler velocity, and antenna dimension.

[0033] The target detection module 103 is used to aggregate the range Doppler spectra generated from multiple data blocks and perform constant false alarm rate detection on the aggregated range Doppler spectra to identify range Doppler velocity cells containing targets. For each detected range Doppler velocity cell, its distance information is recorded.

[0034] The multi-snapshot construction module 104 is used to extract antenna dimension data as a snapshot from the data block corresponding to each window for each detected range-Doppler velocity unit.

[0035] The MUSIC super-resolution angle estimation module 105 is used to perform super-resolution angle estimation based on multiple signal classification MUSIC for multiple snapshots to obtain the angle information of the target reflection point.

[0036] The point cloud generation module 106 is used to combine distance information and angle information to perform coordinate transformation in order to generate a millimeter-wave radar point cloud in Cartesian coordinate system.

[0037] In this embodiment, millimeter-wave radar acquires and receives data, for example, through an analog-to-digital converter (ADC). This received data is then input to a signal preprocessing module 101, where it undergoes range transformation and static background removal before entering a multi-window range-Doppler spectrum generation module 102. The multi-window range-Doppler spectrum generation module 102 slides a window along a slow time dimension and performs a Doppler fast Fourier transform in each window, outputting multiple range-Doppler data blocks. A target detection module 103 aggregates the range-Doppler spectra generated from the multiple data blocks and performs constant false alarm rate (CFAR) detection on the aggregated range-Doppler spectra to obtain range-Doppler velocity elements containing the target. A multi-snapshot construction module 104 extracts antenna data from multiple windows based on the range-Doppler velocity elements, forming a cross-window snapshot sequence. A MUSIC super-resolution angle estimation module 105 constructs a covariance matrix based on the cross-window snapshots and scans the angle spectrum to obtain the azimuth and elevation angles. A point cloud generation module 106 converts the range, azimuth, and elevation angles into a point cloud in Cartesian coordinates.

[0038] The specific working process of the above system 100 can be found by referring to Figure 2 . Figure 2 A flowchart illustrating a sliding window-based millimeter-wave radar point cloud generation method 200 according to an embodiment of this application is shown. It should be noted that... Figure 2 Method 200 in the middle can be understood as a... Figure 1 An illustrative example of the working process of System 100. Therefore, the preceding text, combined with... Figure 1 The relevant descriptions also apply to the following text.

[0039] like Figure 2 As shown, in step S201, the received data of the millimeter-wave radar is acquired, wherein the received data has a fast time dimension, a slow time dimension, and an antenna dimension.

[0040] In step S202, a distance fast Fourier transform is performed along the fast time dimension, and static background removal is performed on the transformed data to obtain distance slow time data.

[0041] In step S203, multiple fixed-length subsequences are extracted along the slow time dimension using a sliding time window, and a Doppler Fast Fourier Transform is performed within each fixed-length subsequence to generate multiple data blocks containing distance, Doppler velocity, and antenna dimension.

[0042] In step S204, the range Doppler spectra generated from multiple data blocks are aggregated, and constant false alarm rate detection is performed on the aggregated range Doppler spectra to identify range Doppler velocity cells containing the target. For each detected range Doppler velocity cell, its distance information is recorded. In step S205, for each detected range-Doppler velocity cell, antenna dimension data is extracted from the data block corresponding to each window as a snapshot, and super-resolution angle estimation based on multiple signal classification (MUSIC) is performed on multiple snapshots to obtain the angle information of the target reflection point.

[0043] In step S206, coordinate transformation is performed by combining distance information and angle information to generate a millimeter-wave radar point cloud in Cartesian coordinate system.

[0044] In this application, the received data from the millimeter-wave radar can be acquired based on millimeter-wave radar waves, for example, through an ADC module of the millimeter-wave radar. This received data can be represented by a three-dimensional array, specifically including a fast time dimension, a slow time dimension, and an antenna dimension. The fast time dimension comes from sampling within a single pulse, the slow time dimension comes from multiple pulses continuously transmitted within a frame, and the antenna dimension comes from the radar antenna array. The purpose of acquiring this received data is to provide raw observation data for subsequent range estimation, Doppler estimation, and angle estimation.

[0045] Next, for the acquired received data, a fast Fourier transform along the fast time dimension can be performed, and static background removal can be performed on the transformed data to obtain the slow time data.

[0046] In some embodiments, the fast time dimension, after undergoing the aforementioned fast Fourier transform, can be converted into a distance dimension, dividing the space into multiple distance cells. The received data includes dynamic target reflection signals. Static background removal includes: calculating the average value along the slow time dimension for each distance cell, and subtracting this average value from the signal of the corresponding distance cell to highlight dynamic target reflection signals in the slow-time distance data. This suppresses stable reflections from static objects in the environment, making dynamic target reflections more prominent in the slow-time distance data.

[0047] Next, multiple fixed-length subsequences can be extracted along the slow time dimension using a sliding time window, and a Doppler Fast Fourier Transform can be performed within each fixed-length subsequence to generate multiple data blocks containing distance, Doppler velocity, and antenna dimension.

[0048] In some embodiments, the extraction process of multiple fixed-length subsequences may include: using a pulse sequence in the slow time dimension as a sliding object, sliding a fixed-length window from the start of the pulse sequence backward, and extracting a fixed-length subsequence each time the window slides, so that the same Doppler velocity unit in different windows corresponds to the same physical Doppler velocity. The length of the fixed-length window is determined by the number of antennas and the number of targets to be resolved. Specifically, when the antenna array has p antennas, the maximum number of targets to be resolved is... The number of windows M does not exceed p-1 and satisfies When the total number of pulses is N, the length of the fixed-length window satisfy This process facilitates the subsequent extraction of multiple antenna snapshots of the same velocity element across windows.

[0049] Next, the range-Doppler spectra generated from multiple data blocks are aggregated, and constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectrum to identify range-Doppler velocity cells containing the target. Specifically, for each detected range-Doppler velocity cell, its range information is recorded.

[0050] In some embodiments, the process of aggregating the range-Doppler spectra generated from multiple data blocks includes: averaging each data block containing range, Doppler velocity, and antenna dimension along the antenna dimension to obtain the range-Doppler spectrum of the corresponding window; and aggregating the range-Doppler spectra corresponding to different windows to form an aggregated range-Doppler spectrum. The aggregation process in this embodiment can improve the signal-to-noise ratio of the aggregated range-Doppler spectrum.

[0051] Furthermore, the aforementioned Constant False Alarm Rate (CFAR) detection is performed on the aggregated range-Doppler spectrum and is used to determine the range index and Doppler velocity index containing the target. The range index is used to obtain the distance to the target reflection point, and the Doppler velocity index is used to locate the antenna dimension data with the same physical Doppler velocity from data blocks corresponding to multiple windows. Thus, based on CFAR detection, the range information of the range-Doppler velocity cells containing the target can be obtained.

[0052] Next, for each detected range-Doppler velocity cell, antenna dimension data is extracted from the data block corresponding to each window as a snapshot, and super-resolution angle estimation based on multiple signal classification (MUSIC) is performed on multiple snapshots to obtain the angle information of the target reflection point.

[0053] In some embodiments, the super-resolution angle estimation process specifically involves: performing autocorrelation processing on each snapshot to obtain the covariance matrix corresponding to that snapshot; averaging the covariance matrices corresponding to multiple snapshots to obtain the received signal covariance matrix used for angle estimation; performing eigenvalue decomposition on the received signal covariance matrix to obtain the noise subspace; and scanning the angle spectrum based on the noise subspace and the steering vector to obtain the angle information of the target reflection point. Averaging the covariance matrices corresponding to multiple snapshots includes: assuming the antenna array received signal in the k-th window is... The total number of windows is M, and the covariance matrix corresponding to the k-th window is... Then the received signal covariance matrix used for super-resolution angle estimation is The guide vector scanning angle spectrum is a two-dimensional azimuth and elevation angle spectrum. The two-dimensional antenna array is located in the xz plane, with the position of the first antenna as the origin, and the coordinates of the i-th antenna are... The phase difference between the i-th antenna and the first antenna is Where λ is the wavelength. and These are the azimuth and elevation angles of the target signal, respectively, and the steering vector is... Angular spectrum This processing overcomes the problem that each velocity unit retains only a single snapshot after the Doppler Fast Fourier Transform of the complete pulse sequence, enabling the MUSIC algorithm to have the time diversity required for decoherence.

[0054] Then, a coordinate transformation is performed by combining the distance and angle information to generate a millimeter-wave radar point cloud in Cartesian coordinates. Specifically, the millimeter-wave radar point cloud can be generated by coordinate transformation using the distance, azimuth, and elevation angles of the target reflection point.

[0055] In the above process, the millimeter-wave radar point cloud generation method 200 based on sliding windows (hereinafter referred to as method 200) specifically includes signal preprocessing, multi-window range Doppler spectrum generation, target detection, multi-snapshot construction, MUSIC super-resolution angle estimation, and coordinate transformation to generate point clouds. Specifically, by employing a sliding window on the slow-time pulse sequence, the complete pulse sequence is divided into multiple overlapping subsequences with temporal order, and a Doppler Fast Fourier Transform is performed within each subsequence. This preserves the temporal diversity across windows while obtaining Doppler velocity resolution, solving the problem that the complete Doppler transform destroys time-series information, leading to ineffective decoherence in super-resolution angle estimation.

[0056] Method 200 first acquires millimeter-wave radar received data, then performs a fast Fourier transform along the fast time dimension to convert the fast time dimension into a range dimension. Since the radar received signal contains both dynamic target reflections and static object reflections from the environment, Method 200 can further calculate an average value along the slow time dimension for each range cell, and subtract the average value from the signal of that range cell to remove static background and highlight dynamic target reflection signals.

[0057] In the multi-window range-Doppler spectrum generation stage, a fixed-length sliding time window is applied to the pulse sequence along the slow time dimension. Unlike performing a Doppler Fast Fourier Transform only once on the complete pulse sequence, Method 200 performs a Doppler Fast Fourier Transform within a fixed-length subsequence truncated in each window, so that each window generates a data block containing range, Doppler velocity, and antenna dimension. Since different windows maintain temporal order, and the same velocity unit corresponds to the same physical Doppler velocity, multiple antenna snapshots can be extracted across windows from the same velocity unit, providing the temporal diversity required for decoherence in the MUSIC algorithm.

[0058] In the target detection phase, the range-Doppler spectrum of each window is obtained by averaging the data blocks along the antenna dimension. The range-Doppler spectra of different windows are then aggregated to improve the signal-to-noise ratio. Finally, constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectrum to identify the range-Doppler velocity cells containing the target. Furthermore, each detected range-Doppler velocity cell containing the target corresponds to a range index and a Doppler velocity index, which are used to locate the corresponding antenna dimension data within each window's data block.

[0059] In the super-resolution angle estimation stage, for each detected range-Doppler velocity cell, antenna dimension data corresponding to the same range index and the same Doppler velocity index are extracted from all windows to form multiple snapshots. Then, autocorrelation processing is performed on each snapshot to obtain the covariance matrix corresponding to each window. The covariance matrices of multiple windows are averaged to obtain the received signal covariance matrix used for MUSIC angle estimation. Eigenvalue decomposition is performed on this covariance matrix to obtain the noise subspace. Combined with the azimuth and elevation steering vectors of the two-dimensional antenna array, the super-resolution angle information of the target reflection point is obtained. Finally, coordinate transformation is performed by combining the range information, azimuth, and elevation angles to generate a millimeter-wave radar point cloud in Cartesian coordinates.

[0060] For details on the specific operation of the aforementioned system 100, please refer to... Figure 3 . Figure 3 A flowchart illustrating another embodiment of the millimeter-wave radar point cloud generation method 300 based on a sliding window of this application is shown. It should be noted that... Figure 3 Method 300 in the text can be understood as a... Figure 1 The example illustration of the working process of System 100 can also be understood as a demonstration of... Figure 2 Further limitations or extensions of Chinese method 200. Therefore, the preceding text, combined with... Figure 1 and Figure 2 The relevant descriptions also apply to the following text.

[0061] like Figure 3As shown, in step S301, the millimeter-wave radar received data is acquired. The millimeter-wave radar received data is a three-dimensional array, containing a fast time dimension, a slow time dimension, and an antenna dimension. For a more detailed description of this processing step, please refer to [reference needed]. Figure 2 This will not be elaborated upon here.

[0062] At step S302, signal preprocessing can be performed. First, a range-fast Fourier transform is performed on the received data along the fast time dimension, converting the fast time dimension to a range dimension, thus dividing the space into multiple range cells. Then, static background removal is performed. For each range cell, its average signal along the slow time dimension is calculated, and the average signal is subtracted from the slow time signal of that range cell. This processing suppresses stable reflections from static objects in the environment, making reflections from dynamic targets more prominent in the slow-time range data.

[0063] In step S303, multiple range-Doppler spectra are generated. (Refer to...) Figure 4 In slow time dimension pulse sequences (e.g.) Figure 4 A fixed-length sliding window is set on (P1-P16). The window slides backward from the beginning of the pulse sequence, extracting a fixed-length subsequence each time it slides (e.g., using window 1-3 to extract a fixed-length subsequence). A Doppler Fast Fourier Transform is performed on each fixed-length subsequence along the slow time dimension (e.g., ...). Figure 4 The Doppler FFT within window 1, window 2, and window 3 is used to convert the slow time dimension of the subsequence into the Doppler velocity dimension. Each window generates a data block containing range, Doppler velocity, and antenna dimension. Then, the average of each data block along the antenna dimension is calculated to obtain the range-Doppler spectrum corresponding to that window (e.g., range-Doppler spectrum 1, range-Doppler spectrum 2, range-Doppler spectrum 3, etc.). Different windows have a temporal order, and the same Doppler velocity unit corresponds to the same physical Doppler velocity, so multiple antenna snapshots of the same velocity unit can be extracted across windows subsequently.

[0064] return Figure 3 In step S304, target detection is performed. The range-Doppler spectra corresponding to multiple windows are aggregated to form an aggregated range-Doppler spectrum. (Reference) Figure 4 Range-Doppler spectra 1, 2, and 3 are aggregated. Based on the target detection module 103 in system 100, constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectrum to identify range-Doppler velocity cells containing target reflections. For each detected range-Doppler velocity cell, its range index is recorded. And Doppler velocity index The distance index is used to determine the distance to the target reflection point, and the Doppler velocity index is used to locate the corresponding velocity unit in each window data block.

[0065] In step S305, a multi-window snapshot is constructed and MUSIC super-resolution angle estimation is performed. (Reference) Figure 4 Based on the multi-snapshot construction module 104 in system 100, the following is executed: For a detected range-Doppler velocity cell, Y(t) is extracted from the data blocks corresponding to the first window to the Mth window. , Antenna dimension data in ,:) format, with each window extracting data as a snapshot, and M windows corresponding to M snapshots (e.g. Figure 4 (Snapshots 1-3 in the image). This processing overcomes the problem that each velocity unit retains only a single snapshot after the Doppler Fast Fourier Transform of the complete pulse sequence, enabling the MUSIC algorithm to have the time diversity required for decoherence.

[0066] Then, MUSIC super-resolution angle estimation is performed based on the MUSIC super-resolution angle estimation module 105 in system 100. In this step, the antenna array is assumed to have p antennas, and the number of target incident signals is q. The antenna array received signal corresponding to the velocity element in the first window is represented as:

[0067] in, Let p×1 be the column vector of received signals. Let be the column vector of the incident signal from the q×1 target. For the noise column vector, It is an array response matrix composed of guide vectors corresponding to different target incident angles.

[0068] refer to Figure 5 Let the time interval between adjacent pulse transmissions be . The Doppler frequency of the i-th target is The phase change introduced by the second window relative to the first window is: .set up The antenna array received signal in the k-th window is represented as:

[0069] Autocorrelation processing is performed on the data in the k-th window to obtain the corresponding covariance matrix:

[0070] in, Let be the incident signal covariance matrix. Let be the noise covariance matrix. Averaging the covariance matrices of the M windows yields:

[0071] Temporal diversity introduced through multi-window phase difference It can satisfy the rank condition for separating the signal subspace and noise subspace in the MUSIC algorithm. When When the rank is equal to the target quantity q, for Perform eigenvalue decomposition; the eigenvectors corresponding to the smaller eigenvalues ​​constitute the noise subspace. .

[0072] Regarding window size design, the window size can be determined based on the minimum number of snapshots required by the MUSIC algorithm. For example, the number of antennas can be p, the number of targets to be resolved in the angular dimension can be q, and the number of windows can be M. Since the MUSIC algorithm requires effective separation of the signal subspace and the noise subspace, the rank of the covariance matrix should reach the target number q. The time delay Td between adjacent windows in cross-window data introduces different phase changes in the Doppler frequencies of different targets. The vector formed by multiple windows... This can form a Vandermonde structure with rank min(M,q). Further, let the maximum number of objects to be analyzed be... ,and The total number of pulses is N, not exceeding p-1. Since the MUSIC algorithm requires effective separation of the signal and noise subspaces, the rank of the covariance matrix should reach the target number q. When M ≥ q, it can be guaranteed that the vectors corresponding to different targets are linearly independent, thus ensuring that the average covariance matrix meets the rank requirement of MUSIC super-resolution angle estimation. Therefore, the minimum window size is equal to the number of targets to be resolved, q. Furthermore, when the antenna array has p antennas, the maximum number of resolvable targets in the angular dimension is p-1, let this number be... If the total number of pulses is N, then it is simultaneously satisfying To maintain a larger window length and improve Doppler velocity resolution, the window length is set to... .

[0073] After obtaining the noise-based subspace, angular spectrum scanning can be performed based on the noise subspace. (Refer to...) Figure 5 For a two-dimensional antenna array, assuming the antenna array is located in the xz plane, with the position of the first antenna as the origin, the coordinates of the second antenna are... The coordinates of the i-th antenna are The wavelength is λ, and the azimuth angle of the target signal is... Pitch angle is Then the phase difference between the i-th antenna and the first antenna is:

[0074] The corresponding two-dimensional guide vector is:

[0075] By scanning and The azimuth and elevation angle spectrum is obtained:

[0076] Based on this angle spectrum, the azimuth and elevation angles of the target reflection point can be obtained. Then, the azimuth and elevation angles are sent to the point cloud generation module 106.

[0077] return Figure 3 In step S306, a point cloud is generated. The distance information obtained in step S304 is combined with the azimuth and pitch angles obtained in step S305, and a coordinate transformation is performed to obtain a three-dimensional point cloud in Cartesian coordinates. The generated point cloud can be used as input for pose recognition, fitness activity recognition, and other point cloud perception tasks.

[0078] Based on the above steps, the sliding window strategy supports Doppler velocity estimation and MUSIC super-resolution angle estimation simultaneously in the slow time dimension, enabling millimeter-wave radar to jointly utilize the three dimensions of range, Doppler velocity, and super-resolution angle to analyze target reflection points, thereby generating a higher density point cloud.

[0079] Furthermore, experimental results show that in point cloud density evaluation, the technical solution of this application, compared with research systems based on range and Doppler velocity and industrial systems based on range and angle, can increase the number of reflection points by 2.16 times, and by 3.99 times compared with the industrial system. For 24GHz, 60GHz, and 77GHz millimeter-wave radars, under the same 250MHz bandwidth configuration, the technical solution of this application can improve the point cloud density.

[0080] In human pose recognition applications, the point cloud obtained by the technical solution of this application is used to identify three poses: lying down, sitting, and standing. The continuous point cloud stream is divided into 2-second long samples, and a PointNet++-based model is used for recognition. Experiments employ cross-user and cross-scene training and testing. Training data comes from three volunteers in a living room scene, and test data comes from two volunteers in a bedroom scene. Results show that the accuracy rate for each pose recognition exceeds 98%, with an overall accuracy rate of 99.28%, higher than the 72.33% of the research system and the 64.85% of the industrial system.

[0081] In the fitness activity recognition application, the point cloud obtained from the proposed technical solution was used to identify five types of activities: boxing, jumping, walking, jumping jacks, and squatting. The continuous point cloud stream was divided into 2-second long samples, and a PointNet++-based model was used for recognition. Experiments employed cross-user and cross-scene training and testing. Training data came from three volunteers in a living room scene, and testing data came from two volunteers in a bedroom scene. Results showed that the overall recognition accuracy supported by the proposed technical solution reached 97.72%, higher than the 65.44% of the research system and the 41.18% of the industrial system.

[0082] Having introduced the method of the exemplary embodiments of this application, the following describes related products of the millimeter-wave radar point cloud generation method based on sliding window of the exemplary embodiments of this application.

[0083] The electronic device involved in this application may include a processor and a memory. The memory stores computer instructions for generating millimeter-wave radar point clouds based on a sliding window. When the computer instructions are executed by the processor, the electronic device performs the following: acquiring received data from the millimeter-wave radar, wherein the received data has a fast time dimension, a slow time dimension, and an antenna dimension; performing a range fast Fourier transform along the fast time dimension and performing static background removal on the transformed data to obtain range slow time data; extracting multiple fixed-length subsequences along the slow time dimension using a sliding time window, and performing a Doppler fast Fourier transform within each fixed-length subsequence to generate multiple data containing range, Doppler velocity, and antenna dimension. The data blocks are aggregated to generate range-Doppler spectra, and constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectra to identify range-Doppler velocity cells containing targets. For each detected range-Doppler velocity cell, its range information is recorded. For each detected range-Doppler velocity cell, antenna dimension data is extracted from the data block corresponding to each window as a snapshot, and super-resolution angle estimation based on multiple signal classification (MUSIC) is performed on multiple snapshots to obtain the angle information of the target reflection point. Finally, coordinate transformation is performed by combining the range information and the angle information to generate a millimeter-wave radar point cloud in Cartesian coordinates.

[0084] Through the above implementation methods, electronic devices can simultaneously support Doppler velocity estimation and MUSIC super-resolution angle estimation in the slow time dimension using a sliding window strategy. This enables millimeter-wave radar to jointly utilize three dimensions—range, Doppler velocity, and super-resolution angle—to resolve target reflection points, thereby generating higher-density point clouds. Furthermore, by employing a sliding window on the slow-time pulse sequence, the complete pulse sequence is divided into multiple overlapping subsequences with temporal order. A Doppler Fast Fourier Transform is performed within each subsequence, thus preserving cross-window temporal diversity while obtaining Doppler velocity resolution. This solves the problem that the complete Doppler transform destroys time-series information, leading to ineffective decoherence in super-resolution angle estimation.

[0085] In some embodiments, the electronic device may specifically include the aforementioned sliding window-based millimeter-wave radar point cloud generation system 100. In some implementation scenarios, system 100 is specifically used with a commercial 24GHz millimeter-wave radar, model ICLagend AD2402, with a starting frequency configured at 24GHz, each frame containing 128 chirp pulses, a bandwidth configured at 250MHz, and an antenna array comprising two transmitting antennas and four receiving antennas, forming an eight-element virtual array. Radar data is transmitted via USB to a laptop computer with an Intel i9-13900HX processor and 32GB of memory. MATLAB is used to parse the received data and perform signal processing to generate point clouds. System 100 can also be used with the 60GHz commercial millimeter-wave radar Calterah RDP-60S244-IBM and the 77GHz commercial millimeter-wave radar Calterah RDP-77S244-ABM to verify cross-frequency band and cross-device adaptability. It should be noted that the technical solution of this application does not limit the specific application scenario.

[0086] In addition, it should be noted that the specific details of the operating method steps of this electronic device are combined with the foregoing. Figures 2-5 The specific implementation methods described are the same or similar, so they will not be elaborated here.

[0087] Furthermore, this application also provides a computer-readable storage medium storing program instructions configured to execute at runtime. Figures 2-5 The method for generating millimeter-wave radar point clouds based on a sliding window is shown.

[0088] Specifically, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0089] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for generating millimeter-wave radar point clouds based on a sliding window, characterized in that, include: Acquire received data from millimeter-wave radar, wherein the received data has a fast time dimension, a slow time dimension, and an antenna dimension; Perform a distance fast Fourier transform along the fast time dimension, and perform static background removal on the transformed data to obtain distance slow time data; Multiple fixed-length subsequences are extracted along the slow time dimension using a sliding time window, and a Doppler Fast Fourier Transform is performed within each fixed-length subsequence to generate multiple data blocks containing distance, Doppler velocity, and antenna dimension. The range-Doppler spectra generated from multiple data blocks are aggregated, and constant false alarm rate detection is performed on the aggregated range-Doppler spectra to identify range-Doppler velocity cells containing targets. For each detected range-Doppler velocity cell, its range information is recorded. For each detected range-Doppler velocity cell, antenna dimension data is extracted from the data block corresponding to each window as a snapshot, and super-resolution angle estimation based on multiple signal classification (MUSIC) is performed on multiple snapshots to obtain the angle information of the target reflection point. as well as By combining distance and angle information, a coordinate transformation is performed to generate a millimeter-wave radar point cloud in Cartesian coordinates.

2. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 1, characterized in that, The fast time dimension is converted into a distance dimension by the distance fast Fourier transform, dividing the space into multiple distance units. The received data includes dynamic target reflection signals, and the static background removal includes: For each distance cell, calculate its average value along the slow time dimension; The average value is subtracted from the signal of the corresponding distance unit to highlight the dynamic target reflection signal in the slow-time distance data.

3. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 1, characterized in that, The extraction of multiple fixed-length subsequences along the slow time dimension using a sliding time window includes: Treat the pulse sequence in the slow time dimension as the sliding object; A fixed-length window is used to slide backward from the beginning of the pulse sequence; Each slide extracts a fixed-length subsequence; Make the same Doppler velocity element in different windows correspond to the same physical Doppler velocity.

4. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 3, characterized in that, The length of the fixed-length window is determined by the number of antennas and the number of targets to be analyzed.

5. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 1, characterized in that, The aggregation of the range Doppler spectra generated from multiple data blocks includes: For each data block containing range, Doppler velocity, and antenna dimension, the average is calculated along the antenna dimension to obtain the range-Doppler spectrum of the corresponding window; The range-Doppler spectra corresponding to different windows are aggregated to form an aggregated range-Doppler spectrum.

6. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 1, characterized in that, The constant false alarm rate (CFAR) detection is performed on the aggregated range-Doppler spectrum and is used to determine the range index and Doppler velocity index containing the target. The range index is used to obtain the distance to the target reflection point, and the Doppler velocity index is used to locate the antenna dimension data with the same physical Doppler velocity from data blocks corresponding to multiple windows.

7. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 1, characterized in that, The super-resolution angle estimation based on multiple signal classification (MUSIC) for multiple snapshots includes: Perform autocorrelation processing on each snapshot to obtain the covariance matrix corresponding to that snapshot; The covariance matrices corresponding to multiple snapshots are averaged to obtain the received signal covariance matrix used for angle estimation. Eigenvalue decomposition is performed on the covariance matrix of the received signal to obtain the noise subspace; The angle information of the target reflection point is obtained by scanning the angle spectrum based on the noise subspace and the guide vector.

8. The millimeter-wave radar point cloud generation method based on a sliding window according to claim 1, characterized in that, The millimeter-wave radar point cloud is generated by coordinate transformation of the distance, azimuth, and elevation angle of the target reflection point.

9. A millimeter-wave radar point cloud generation system based on a sliding window, characterized in that, include: The signal preprocessing module is used to acquire the received data from the millimeter-wave radar, perform a range fast Fourier transform along the fast time dimension, and perform static background removal on the transformed data to obtain range slow time data, wherein the received data has the fast time dimension, slow time dimension and antenna dimension. The multi-window range-Doppler spectrum generation module is used to extract multiple fixed-length subsequences along the slow time dimension using a sliding time window, and to perform Doppler fast Fourier transform within each fixed-length subsequence to generate multiple data blocks containing range, Doppler velocity, and antenna dimension. The target detection module is used to aggregate the range-Doppler spectra generated from multiple data blocks and perform constant false alarm rate detection on the aggregated range-Doppler spectra to identify range-Doppler velocity cells containing targets. For each detected range-Doppler velocity cell, its distance information is recorded. The multi-snapshot construction module is used to extract antenna dimension data as a snapshot from the data block corresponding to each window for each detected range-Doppler velocity unit; The MUSIC super-resolution angle estimation module is used to perform super-resolution angle estimation based on multiple signal classification MUSIC for multiple snapshots to obtain the angle information of the target reflection point; The point cloud generation module is used to combine distance and angle information to perform coordinate transformation in order to generate millimeter-wave radar point clouds in Cartesian coordinates.

10. An electronic device, characterized in that, include: The memory stores computer instructions for generating millimeter-wave radar point clouds based on a sliding window. A processor that executes the computer instructions to cause the electronic device to perform the millimeter-wave radar point cloud generation method based on a sliding window according to any one of claims 1 to 8.

Images