Target detection method, integrated circuit, sensor, device, and medium

Abstract

A target detection method, an integrated circuit, an electromagnetic wave sensor, a device, and a computer readable storage medium, relating to the technical field of target detection. The target detection method comprises: performing FFT processing on an echo signal to generate a feature vector of the echo signal; and processing the feature vector by means of a machine learning model so as to implement target detection in a region of interest.

Description

Target detection method, integrated circuit, sensor, device and medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number "202310913653.8" and application date of July 24, 2023, and claims the priority of the above-mentioned Chinese patent application. The entire content of the above-mentioned Chinese patent application is hereby incorporated into this application by introduction. Technical Field

[0003] The embodiments of the present application relate to the field of target detection technology, and in particular to a target detection method, integrated circuit, electromagnetic wave sensor, device, and computer-readable storage medium. Background Art

[0004] When detecting targets in short-range, enclosed or relatively enclosed spaces, such as within a car cabin, strong static clutter and multipath can lead to technical issues such as low detection rates, high false alarm rates, and inaccurate angle estimation. This is especially true for scenarios where children are left behind in a car, where their RCS (Radar Cross Section) is low, making detection more challenging. Furthermore, rule-based seating area determination often relies on the measurement of spatial geometry, the design of appropriate rules, and the optimization of various parameters, complicating equipment deployment and commissioning.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a method, integrated circuit, electromagnetic wave sensor, device, and computer-readable storage medium for target detection, which can improve the accuracy and efficiency of target detection.

[0007] According to some embodiments of the present application, a first aspect of the embodiments of the present application provides a target detection method, comprising: performing FFT processing on an echo signal to generate a feature vector of the echo signal; and processing the feature vector based on a machine learning model to achieve target detection in an area of ​​interest.

[0008] According to some embodiments of the present application, the second aspect of the embodiments of the present application further provides an integrated circuit, comprising a radio frequency module, an analog signal processing module and a digital signal processing module connected in sequence; wherein the radio frequency module is used to generate a radio frequency transmit signal and receive a radio frequency receive signal; the analog signal processing module is used to down-convert the radio frequency receive signal to obtain an intermediate frequency signal; the digital signal processing module is used to perform analog-to-digital conversion on the intermediate frequency signal, and based on this, implement the target detection method described in the first aspect of the embodiments of the present application.

[0009] According to some embodiments of the present application, the third aspect of the embodiments of the present application further provides an electromagnetic wave sensor, comprising: a carrier; the integrated circuit as described in the second aspect of the embodiments of the present application, which is arranged on the carrier; an antenna, which is arranged on the carrier, or the antenna and the integrated circuit are integrated into a single device and arranged on the carrier; wherein the integrated circuit is connected to the antenna for transmitting radio frequency transmission signals and / or receiving radio frequency reception signals.

[0010] According to some embodiments of the present application, the fourth aspect of the embodiments of the present application further provides a device, including: a device body; and an electromagnetic wave sensor as described in the third aspect of the embodiments of the present application, which is arranged on the device body; wherein the electromagnetic wave sensor is used for target detection and / or communication to provide reference information for the operation of the device body. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1 is a flow chart of a target detection method provided in an embodiment of the present application;

[0012] FIG2 is a schematic diagram of a flow chart of a target detection method provided in an embodiment of the present application;

[0013] FIG3 is a flow chart of a target detection method provided in an embodiment of the present application;

[0014] FIG4 is a flow chart of a target detection method provided in an embodiment of the present application;

[0015] FIG5 is a flow chart of a target detection method provided in an embodiment of the present application;

[0016] FIG6 is a schematic diagram of the division of the region of interest involved in the target detection method provided in an embodiment of the present application;

[0017] FIG7 is a schematic diagram of data rearrangement involved in the target detection method provided in an embodiment of the present application;

[0018] FIG8 is a schematic diagram of the structure of a deep learning model involved in the target detection method provided in an embodiment of the present application;

[0019] FIG9 is a flow chart of a target detection method provided in an embodiment of the present application;

[0020] FIG10 is a schematic diagram of a flow chart of a target detection method provided in an embodiment of the present application;

[0021] FIG11 is a schematic diagram of a flow chart of a target detection method provided in an embodiment of the present application;

[0022] FIG12 is a schematic diagram of a flow chart of a target detection method provided in an embodiment of the present application;

[0023] 13 to 22 are schematic diagrams of a target detection method provided in another embodiment of the present application;

[0024] FIG23 is a schematic diagram of data rearrangement involved in the target detection method provided in an embodiment of the present application;

[0025] FIG24 is a flow chart of a target detection method provided in an embodiment of the present application;

[0026] FIG25 is a flow chart of a target detection method provided in an embodiment of the present application;

[0027] Figures 26A-26B are schematic diagrams of the effects of the target detection method provided in the embodiments of the present application. DETAILED DESCRIPTION

[0028] The present disclosure describes a plurality of embodiments, but this description is exemplary rather than restrictive, and it will be apparent to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure. Although many possible feature combinations are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0029] The present disclosure includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features, and elements of the present disclosure may also be combined with any conventional features or elements to form a unique inventive solution defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other inventive solutions to form another unique inventive solution defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this disclosure may be implemented individually or in any appropriate combination. Therefore, the embodiments are not subject to other limitations except for the limitations set forth in the appended claims and their equivalents. In addition, various modifications and changes may be made within the scope of protection of the appended claims.

[0030] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to performing their steps in the order written, and those skilled in the art can readily understand that these orders can be changed and still remain within the spirit and scope of the embodiments of the present application.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art of the present disclosure. The terms used in the specification of the present disclosure are only for describing the embodiments and are not intended to limit the present disclosure.

[0032] When detecting targets in sealed or relatively sealed spatial areas, in some optional embodiments, target detection can be achieved by performing regional judgment based on single-frame fast and slow time processing, CFAR (constant false alarm) detection, and angle measurement to obtain a point cloud, or by performing DoA (direction of arrival estimation) operations using algorithms such as Capon after single-frame fast time processing to form an RA (Range-Azimuth) map, and then extracting features such as power in a predetermined area for identification and judgment to achieve target detection, or by filtering the received echo phase and then detecting human breathing and heartbeat to achieve target detection. For example, when target detection is achieved by performing regional judgment based on single-frame fast and slow time processing, CFAR detection, and angle measurement to obtain a point cloud, after performing 1D-FFT (such as range-dimensional FFT) processing on the echo signal, time correlation analysis can be performed based on the echo data to achieve human target identification, and then position judgment can be performed by combining point cloud information to ultimately achieve the purpose of in-cabin target detection.

[0033] In some embodiments, the target detection process is shown in FIG1 . After the radar (taking target detection of a frequency modulated continuous wave (FMCW) radar) performs analog-to-digital conversion, sampling, and other operations to process the echo signal received by the antenna, it will be further processed by a main control module (for example, which can be implemented based on a microcontroller unit (MCU)) and a baseband (BB) module (for example, which can be implemented based on a baseband chip) to achieve target detection: Range DC removal and Range FFT (also expressed as 1D-FFT or fast time FFT) are performed in sequence on the baseband module. The baseband module then sends the signal to the main control module for multi-frame accumulation on the main control module. When the data accumulates to a preset number of frames, such as 128 frames, the main control module moves the accumulated data from the CPU static random-access memory (SRAM) to the baseband module, and Doppler DC removal is continued based on the baseband module. The system then performs processing such as DC removal, inter-frame Fourier transform (Frame FFT), and inter-channel accumulation (such as non-coherent integration and correlation accumulation). In the baseband module, the system then performs constant false-alarm rate (CFAR) detection based on the noise obtained by noise estimation (such as using a variance-based noise estimator), such as peak-based CFAR (detection with peak selection). Finally, it performs angle detection and target classification to obtain the target's distance, speed, and / or angle information.

[0034] The relevant steps in Figure 1 can be implemented as follows:

[0035] Range-dimensional DC removal targets the data corresponding to each chirp in each frame. Specifically, this removal involves averaging the data collected by each chirp and each receive (RX) channel along the fast time dimension, then subtracting the DC component from all sampling points in each RX channel.

[0036] Range-dimensional FFT: This can be achieved by windowing the data after DC removal in the range dimension and then performing 1D-FFT.

[0037] It is important to note that during target detection, each chirp data in each frame of data obtained by radar through ADC or sampling based on the echo signal is processed as described above and sent to the main control module. This data can be sent to the main control module by moving the data from Direct Memory Access (DMA) to the CPU SRAM.

[0038] Doppler DC removal: This can be achieved by calculating the complex average along the frame dimension of the multi-frame data accumulated on the main control module (i.e., averaging the data of different frames in the same distance unit (bin) of the same channel), obtaining the average value of different channels and different distances as the DC component of the corresponding distance unit of the corresponding channel, and subtracting the respective DC component from each distance unit of each channel.

[0039] Inter-frame Fourier transform: This can be achieved by windowing the data after removing the Doppler DC along the frame dimension and performing a 2D-FFT along the frame dimension. This will generate range-Doppler information for each transmit and receive channel. For example, in some embodiments, this can be achieved by performing a range-dimensional FFT on the echo signal, sliding-windowing the result obtained from the range-dimensional FFT, and performing a 2D FFT on the windowed data to obtain a range-Doppler spectrum. The length of the sliding window corresponds to a duration of the same order of magnitude as the period of the target's periodic motion. By performing digital signal processing on multiple frames of data with similar periods of the target's periodic motion, target detection can be better performed by combining the characteristics of the target's motion period, avoiding errors and omissions while also facilitating the elimination of multipath and other influences, thereby improving target detection accuracy.

[0040] Incoherent accumulation: can be achieved through the following expression:

[0041] or,

[0042] where P(r,v) and A(r,v) are the power and amplitude at the rth range unit and the vth Doppler unit, respectively, and s(r,t,a,v) is the complex value obtained by inter-frame Fourier transform of the signal echo of the rth range unit and the vth Doppler unit on the tth transmit channel and the ath receive channel.

[0043] The constant false alarm detection based on the noise floor obtained by noise estimation has been described in the existing CFAR technology and will not be described in detail here.

[0044] Angle detection can be achieved by performing digital beam forming (DBF) and DOA in azimuth, as well as DBF and DOA in elevation, on the target points obtained by CFAR. DBF and DOA are already described in existing DBF and DOA technologies and will not be elaborated on here.

[0045] Target classification: This can be achieved by counting the target points detected in each frame (or preset data volume, i.e. each time) according to a preset (or divided) area unit design, and the counting results can be judged according to preset rules to obtain whether there is a physical target of interest (such as an adult, child, infant or pet, etc.) in this processing, and determine its location area.

[0046] It should be noted that FIG1 is merely an example for a radar having a baseband module and a main control module. That is, operations such as frame data accumulation and target classification (which can be implemented using Region HIST or classification) are performed in the main control module, while other operations, such as range-dimensional DC removal and Doppler-dimensional DC removal, are performed in the baseband module. In other embodiments, frame data accumulation may be performed in the baseband module, and Doppler-dimensional DC removal may be performed in the main control module.

[0047] It should also be noted that Figure 1 only uses the cache of 128 frames of data as an example for illustration. In other embodiments, 56 frames of data or 32 frames can also be cached. The embodiments of the present application do not limit this and can be determined based on demand or hardware support capabilities. In this way, the cached multi-frame data will continue to undergo Doppler dimension DC removal and inter-frame FFT to obtain a range-Doppler (RD) diagram.

[0048] It should also be noted that in the process shown in Figure 1, some steps can be skipped, such as distance-dimensional DC removal or distance-dimensional DC removal or windowing; some steps can be simplified, such as non-coherent accumulation can be skipped, and only CFAR detection is performed on one of the channels, or only some channels are selected for non-coherent accumulation, and CFAR detection is performed on these channels.

[0049] Furthermore, on the basis of the above scheme, the embodiment of the present application further proposes to combine with a deep learning model for processing. By using a deep learning model to deeply mine the target information, a more accurate scheme for target detection in spatial areas such as car cabins, indoors, and factory buildings can be achieved. While effectively improving the detection rate and reducing the number of false alarm targets, it can also greatly improve the accuracy of angle estimation and other effects, thereby enabling special targets or weak targets such as infants in the cabin to be accurately detected, and realizing applications such as CPD (Child Presence Detection, child detection in the car), SBR (Safety Belt Reminder, seat belt reminder device) and the like. It should be noted that the deep learning model can be combined with the above embodiment in different ways, and its different combinations will be explained below.

[0050] In some embodiments,

[0051] As shown in Figure 2, the target detection method includes the following steps:

[0052] Step 101 : Perform FFT processing and beamforming processing on the echo signal to generate a feature vector of the echo signal, wherein the feature vector includes energy characteristics representing the echo signal at different distance units.

[0053] Step 102: Process the feature vector based on the deep learning model to achieve target detection in the region of interest.

[0054] In this way, through beamforming processing, the signal-to-clutter ratio of human targets to static strong clutter is effectively improved, and feature vectors are effectively constructed for empty vehicles (with or without interference), scenes with adults, and scenes with children. The target information contained in the feature vectors is deeply mined through a deep learning model to achieve greater precision. This solution can realize target detection in spatial areas such as car cabins, indoors, and factory buildings. While effectively improving the detection rate and reducing the number of false alarm targets, it can also significantly improve the accuracy of angle estimation and other effects. In turn, special targets or weak targets such as infants and young children in the cabin can be accurately detected, realizing applications such as CPD and SBR. At the same time, the introduction of the deep learning module can also reduce the required digital signal processing. For example, after extracting the features, there is no need to perform CFAR, DOA, and other processing.

[0055] To better understand the embodiment shown in FIG. 2 , its steps will be described below.

[0056] In step 101, the implementation of FFT processing and beamforming is not limited. It is understood that, since beamforming essentially focuses the signal's directional gain (which reflects energy) in a specified direction, beamforming effectively extracts energy features. In other words, different implementations of FFT processing and beamforming do not affect the extraction of energy features, and thus do not affect the generation of feature vectors representing the energy features of the echo signal at different range units.

[0057] As can be seen from the process shown in Figure 1, the target detection process performs beamforming processing to achieve angle estimation, and subsequent processing such as target classification is required to output intuitive and accurate results. In the embodiment shown in Figure 2, beamforming processing is used to generate feature vectors, and the subsequent angle estimation, target classification, etc. are replaced by deep learning model processing, which is more efficient. Moreover, since the deep learning model does not directly use time series data as input, but further extracts target information from the echo signal through FFT processing and beamforming processing to form a feature vector with higher dimension and richer information, the deep learning model can better perform analysis and processing, and output more accurate results, which can improve the accuracy and efficiency of target detection and achieve more accurate in-cabin target detection and recognition. Among them, since the noise and target reflection signals have different energy characteristics, the use of feature vectors that represent energy characteristics will help the deep learning model to better perform target detection and output more accurate and reliable results.

[0058] It is understandable that the application of the deep learning model is to directly output the detection results instead of processing such as CFAR and angle estimation. That is, based on the process shown in Figure 1, beamforming is performed after the inter-frame FFT, and the correlation accumulation, CFAR, angle estimation and other processes are replaced by the processing of the deep learning model. That is, in some embodiments, as shown in Figure 3, FFT processing and beamforming processing of the echo signal can be achieved by the following steps:

[0059] Step 1011: Perform FFT processing on the echo signal to generate a range-Doppler spectrum.

[0060] Step 1012: Perform beamforming processing according to the range-Doppler spectrum.

[0061] It is also understood that, as shown in FIG1 , FFT processing includes range-dimensional FFT and inter-frame FFT. Beamforming processing can be supported after the range-dimensional FFT, without waiting for the inter-frame FFT. Furthermore, as previously described, energy features can still be acquired and feature vectors generated regardless of the order in which beamforming processing is performed. Therefore, in some embodiments, as shown in FIG4 , performing FFT and beamforming on an echo signal can also be implemented through the following steps:

[0062] Step 1013, performing range FFT processing on the echo signal;

[0063] Step 1014: performing beamforming processing on the result obtained by the range FFT processing;

[0064] Step 1015: Sliding windowing is performed on the beamforming result, and 2D FFT is performed on the windowed data. Optionally, the 2D-FFT here may also be a sliding window 2D-FFT.

[0065] It can also be understood that after the distance dimension FFT processing, beamforming processing is already supported, and the subsequent inter-frame FFT is to better detect targets through digital signal processing. The deep learning model has the ability to learn implicit features. Therefore, even if inter-frame FFT is not used, good target detection results can still be obtained, and good results can still be achieved without inter-frame FFT. Based on this, in some embodiments, as shown in Figure 5, FFT processing and beamforming processing of the echo signal can be achieved by the following steps:

[0066] Step 1016, performing range FFT processing on the echo signal;

[0067] Step 1017: Perform beamforming processing on the result obtained by the range FFT processing.

[0068] It should be noted that the relevant processing (including distance FFT, frame data accumulation, inter-frame FFT, etc.) performed in the embodiments shown in Figures 3 to 5 has been described in the embodiment shown in Figure 2 and will not be repeated here.

[0069] Of course, the above is only an example of the implementation of FFT processing and beamforming processing. In some embodiments, other suitable methods can be used for processing, which will not be listed here one by one.

[0070] It will be appreciated that in the embodiments shown in Figures 4 and 5 , beamforming is performed based on the results of range-dimensional FFT processing, while in the embodiment shown in Figure 3 , beamforming is performed based on the results of inter-frame FFT processing. Therefore, the implementation of beamforming differs. Specifically, in the embodiments shown in Figures 4 and 5 , beamforming is implemented in the range dimension, i.e., beamforming accumulation is performed at different range units. In the embodiment shown in Figure 3 , beamforming can be implemented in both the range and Doppler dimensions, i.e., beamforming accumulation is performed at different Doppler units within different range units. For ease of understanding, the following description will use the implementation of beamforming in the embodiment shown in Figure 3 as an example.

[0071] In some examples, beamforming based on the range-Doppler spectrum obtained by inter-frame FFT (taking Nc point beamforming as an example) can be implemented using the following expression:

[0072] Where P(r,v,b) is the energy characteristic of the vth Doppler unit of the rth range unit based on beam b, sv(c,b) is the steering vector of beam b on virtual channel c, x(c,r,v) is the data corresponding to the vth Doppler unit of the rth range unit on virtual channel c in the range-Doppler map, N c is the total number of beams.

[0073] Correspondingly, when beamforming is performed based on data obtained from the range-dimensional FFT, x(c, r, v) in the above expression is replaced with x(c, r), i.e., the Doppler dimension is not included. Of course, in other embodiments, other methods can also be used, which will not be detailed here.

[0074] Regarding the steering vector of the beam used in beam synthesis, the embodiments of the present application do not limit it. It can be specified by the user, or it can be automatically generated according to the region of interest, etc. For example, in some embodiments, beam synthesis can be achieved by: determining the target direction pointing to the region of interest and generating a steering vector based on the target direction; performing beam synthesis based on the steering vector. Among them, regarding the determination of the target direction, the embodiments of the present application do not limit it. For example, in some examples, determining the target direction pointing to the region of interest includes: determining the target direction based on the azimuth and pitch angles of the region of interest relative to the radar. In other examples, determining the target direction pointing to the region of interest includes: obtaining measured target data in the region of interest; clustering based on the measured target data, so as to use the azimuth and pitch angles of the cluster center points in each region of interest as the target direction.

[0075] For example, in the vehicle interior shown in Figure 6, consider the scenario requirement of determining whether there are people in six areas: the three rear seats and the aisle in front of the three seats. If there are people, determine whether they are adults or children. A 6-point DBF can be performed, that is, six target directions are determined, pointing to the three seats and three aisles shown in Figure 6. In this case, P(r, v, b) corresponding to six beams will be obtained. Of course, other directions can also be added, such as adding three auxiliary beams pointing to the other three areas. In this case, a 9-point DBF is performed, and P(r, v, b) corresponding to nine beams is obtained.

[0076] The generation of the steering vector can be achieved through the following expression:

[0077] sv(c,b)=exp(1j·2π / λ·(x ccosφ b sinθ b +z c sinφ b )),

[0078] Where sv(c,b) is the steering vector of beam b on virtual channel c, λ is the central wavelength of the radar detection signal, and x c and z c are the horizontal and vertical positions of the antenna corresponding to the virtual channel c, θ b is the azimuth angle corresponding to the target direction, φ b is the pitch angle corresponding to the target direction.

[0079] It should be noted that since the radar's coverage is usually larger than the range of the user's target detection, target detection usually occurs in the user's area of ​​interest. Based on this, in the embodiments of this application, target detection in the area of ​​interest is mainly used as an example for description. However, this does not mean that the target detection method provided in the embodiments of this application can only be used for the area of ​​interest. For example, it can also be used for all detectable areas, etc., and will not be described in detail here or later.

[0080] It is also understandable that the input of the deep learning model will also affect the accuracy of the deep learning model. Considering that the target information in the echo signal also has characteristics in a certain arrangement order. Therefore, in order to help the deep learning model better process the feature vector. The energy features in the feature vector can also be arranged. Based on this, in some embodiments, the feature vector of the echo signal can be generated in the following way: according to preset rules, the energy features obtained by FFT processing and beamforming processing of the echo signal are rearranged to obtain the feature vector. In this way, by rearranging, the correlation of the data in the feature vector is strengthened, which is conducive to better detection by the model.

[0081] It should be noted that the embodiments of the present application do not limit the specific rules of rearrangement. For example, in some embodiments, the preset rules include: data with the same first dimension are arranged continuously in the changing direction of the second dimension, and the whole is arranged continuously in the changing direction of the first dimension, wherein the first dimension and the second dimension are different dimensions in the beam dimension and the distance dimension, respectively. At this time, it can correspond to the situation where only energy characteristics on the distance dimension are generated. For example, in some embodiments, the preset rules include: data with the same third dimension and the same fourth dimension are arranged continuously in the changing direction of the fifth dimension, data with the same third dimension are arranged continuously in the changing direction of the fourth dimension, and the whole is arranged continuously in the changing direction of the third dimension, wherein the third dimension, the fourth dimension and the fifth dimension are different dimensions in the beam dimension, the distance dimension and the Doppler dimension, respectively. At this time, it can correspond to the situation where energy characteristics on the distance dimension and the Doppler dimension are generated.

[0082] For ease of understanding, the preset rules for rearrangement will be described below with reference to the rearranged feature vectors shown in FIG. 7 .

[0083] As shown in Figure 7, the energy features are arranged in physical memory continuously along the v dimension, followed by the r dimension, and finally the b dimension. In the figure, v(0) and v(Nv) represent the 0th and Nvth Doppler bins (a total of Nv+1 Doppler bins), r(0) and r(Nr) represent the 0th and Nrth range bins (a total of Nr+1 range bins), and b0 represents the 0th beam, with a total of N beams. This array of (Nv+1)×(Nr+1)×N energy features is the feature vector input to the deep learning model.

[0084] It can also be understood that, as mentioned above, target detection is implemented based on hardware. Therefore, in order to adapt to the capabilities of the hardware, data processing can also be performed to adapt to the capabilities of the hardware.

[0085] For example, in some embodiments, generating a feature vector for an echo signal can be achieved by performing logarithmic normalization on the energy features obtained by performing FFT processing and beamforming on the echo signal. Logarithmic normalization converts the data from floating-point numbers to fixed-point numbers, thereby improving subsequent processing efficiency and, in turn, improving target detection efficiency.

[0086] For example, in some embodiments, logarithmic normalization of energy features obtained by performing FFT processing and beamforming on echo signals can be achieved by taking the logarithm of the energy features obtained by performing FFT processing and beamforming on the echo signals, and linearly normalizing the logarithm using preset upper and lower bounds. Specifically, based on logarithmic normalization, upper and lower bounds, as well as linear parameters, are introduced to further optimize the data format, making it more convenient to generate feature data for subsequent use.

[0087] The energy features obtained by performing FFT processing and beamforming on the echo signal are logarithmized, and the logarithm result is linearly normalized using preset upper and lower bound constraints, which is achieved through the following expression:

[0088] Among them, P norm (r, v, b) is the logarithmic normalization result, P(r, v, b) is the energy characteristic of beam b at range unit r and Doppler unit v, a0, b0, P max and P min All are preset parameters.

[0089] It should be noted that the embodiments of the present application do not limit the specific values ​​of the above-mentioned preset parameters. In some possible embodiments, b0 can be set to the mean or median of all possible log2(P(r,v,b)). Alternatively, in some embodiments, a0 is the difference between the 95% quantile and the 5% quantile of all possible log2(P(r,v,b)) values ​​divided by 256. In this way, P norm (r, v, b) are normalized to between -127 and 127, which is the range of fixed-point numbers.

[0090] It should be noted that the above is only an example of the processing of relevant data. In some embodiments, other processing methods can also be used. For example, when taking the logarithm, the logarithm of 10 is taken instead of 2. Here, the logarithm of 2 can be taken to normalize the data to -127~127 through the above expression. Other logarithm taking methods or other processing methods can also achieve similar effects. Alternatively, instead of quantization, conversion into double-precision floating point or single-precision floating point, such as int32, int16, int8 or uint32, uint16, uint8, etc., can also improve subsequent processing efficiency. I will not go into details here.

[0091] Among them, regarding the normalization embodiments provided in the above embodiments, their implementation can be implemented in various forms. For example, the complex data can be transferred to the MCU Mem via DMA and then performed in the MCU. It can also be implemented using the hardware engine in the BB and then directly transferred to the MCU Mem via DMA. It can also be partially implemented in the BB and transferred to the MCU Mem via DMA and then the remaining calculations can be performed in the MCU (such as the modulo square operation and the logarithm operation of 2 can be implemented in the BB, and subtraction, division and saturation truncation can be implemented in the MCU), etc., which will not be described in detail here.

[0092] In step 102, there is no limitation on the deep learning model used, which can be any deep learning model, such as Convolutional Neural Networks (CNN) or Transformer, etc., which will not be described in detail here.

[0093] As for the processing process and result output of the deep learning model, it can be understood that it can be adjusted by setting the structure of the deep learning model and training the model.

[0094] For example, in some embodiments, the target detection result is a confidence matrix, and the elements in the confidence matrix correspond to indicating at least one of the following information: the confidence that the target is detected, the confidence that the target exists in the corresponding region of interest, the confidence that an adult target exists in the corresponding region of interest, and the confidence that a child target exists in the corresponding region of interest.

[0095] Also, in some embodiments, the structure of the deep learning model is a CNN with three layers of convolution kernels, and the type, input and output size of each layer are shown in Figure 8. At this time, the input feature dimension of the deep learning model is 6×64×20, corresponding to 6 beam pointing, 64 Doppler units and 20 range units. Each convolution operation includes a two-dimensional convolution layer, a batch normalization layer, a ReLu nonlinear activation layer and a max pooling layer. After performing three convolution operations, the resulting 128×2 feature vector is flattened to one dimension to obtain a 256×1 one-dimensional feature vector, and after random inactivation (Drop out), it passes through a 256×10 fully connected layer to obtain the detection result. For example, whether there is someone in the car, and if so, the confidence level of whether there are adults and children in each seat and aisle in the car.

[0096] At this point, it can be understood that, based on the output of the deep learning model, further processing can be performed and then output to the user. For example, taking the interior space shown in Figure 6 as an example, assume that the deep learning model outputs the following: a 1×10 vector yi formed by the confidence levels of whether there is a person in the car, whether there are children in the three seats, whether there are children in the three aisles, and whether there are adults in the three seats. At this point, the following judgment can be made: if yi(1)>0, it is determined that there is a person in the car without considering the other bits of yi, or if yi(k)=0, k=1, 2, ..., 10, it is also determined that there is no one in the car; if there is k=2, 3, ..., 10, so that yi(k)>0, it is determined that there is a child or an adult in the corresponding area; if the confidence levels of the presence of adults and children in seat A and aisle A are both greater than 0, it is determined that there is only an adult in seat A, and so on for seat B and aisle B and seat C and aisle C. If yi = [-0.1, 7.2, -5, -7.2, -1.5, -1.9, -3.1, -6.4, -6.2, -5.5], it is determined that there is a child in seat A. If yi = [-0.1, -5, -4.2, 7.2, -1.5, -1.9, -3.1, 6.4, -6.2, -5.5], it is determined that there is a child in aisle C and an adult in seat A. If yi = [-0.1, -5, -4.2, 7.2, -1.5, -1.9, -3.1, -6.4, -6.2, 5.5], it is determined that there is only an adult in seat C. If yi = [0.1, -5, -4.2, 7.2, -1.5, -1.9, -3.1, 6.4, -6.2, -5.5], it is determined that the car is empty. At this point, it can be determined that there are multiple people in the car, and false alarms can be greatly reduced, but the risk of missed detection is slightly increased.

[0097] Regarding the training method of the deep learning model, in some embodiments, as shown in FIG9 , before processing the feature vector based on the trained deep learning model to obtain the target detection result output by the deep learning model, the method further includes the following steps:

[0098] Step 103 obtains a training feature vector, wherein the training feature vector carries a one-hot encoding label and is used to represent energy features of the signal at different distance units.

[0099] Step 104: Process the training feature vector using a preset data enhancement method to obtain a training set. The preset data enhancement method includes at least one of the following: randomly adding white noise, flipping the data along the Doppler dimension, and translating the data along the distance dimension.

[0100] Step 105: Train the deep learning model based on the training set.

[0101] To facilitate those skilled in the art to better understand the above embodiments, the training of a deep learning model is illustrated below.

[0102] First, according to the above method of obtaining feature vectors, the training feature vectors are obtained, and the multiple processing data of multiple groups of experiments are accumulated to construct the input feature data set. Among them, each element P i It is the aforementioned (Nv+1)×(Nr+1)×N array.

[0103] Then, according to the experimental conditions Marking, that is, P i Generate the corresponding label L i , obtain the labeled input feature dataset At this time, the generated label L i One-hot encoding can be used. For example, in the car scene shown in Figure 6, if we consider judging whether there are people in the three rear seats and three aisle areas, and at the same time consider classifying adults and children, and do not consider the situation where adults squat in the three aisle areas, then there are a total of 10 classification problems. i is a 1×10 vector, with values ​​of 0 or 1. i The first bit indicates whether there is anyone in the car. If no one, the bit is 1, if there is someone, the bit is 0; the 2nd to 4th bits indicate whether there is a child in seat A-seat C. If yes, the bit is 1, if not, the bit is 0; the 5th to 7th bits indicate whether there is a child in aisle A-aisle C. If yes, the bit is 1, if not, the bit is 0; the 8th to 10th bits indicate whether there is an adult in seat A-seat C. If yes, the bit is 1, if not, the bit is 0. If there is no one in the car, the corresponding L i is [1,0,0,0,0,0,0,0,0,0]; if there is an adult in seat A, then the corresponding value is L i is [0,0,0,0,0,0,0,1,0,0]; if there is a child in aisle A and an adult in seat B, then the corresponding L i is [0,0,0,0,1,0,0,0,1,0]. Note that L i The first digit of the number, which indicates whether there is someone in the car, can also be omitted, i.e. L i It is a 1×9 vector. When it is all zero, it means there is no one in the car.

[0104] Then, The enhanced dataset is obtained by performing data enhancement by randomly adding white noise, randomly flipping along the Doppler dimension, and randomly shifting along the distance dimension. As a result, the number of samples in the enhanced dataset is greatly increased compared to the original dataset. There are 5,000 samples of hollow cars, children, and adults, and 10 rounds of data augmentation can be performed to obtain an augmented dataset of 50,000 samples of each type. (You can choose to include or not include the original dataset ). Among them, you can also Random sorting, taking α=0.3 ratio As a validation dataset The remaining data is used as the training dataset

[0105] Then, deep learning model training is performed as follows:

[0106] Build an uninitialized CNN model and randomly initialize the parameters of each layer;

[0107] The following steps are repeated 20 times (each time is called an epoch):

[0108] Will Randomly divide into N b batches, each batch is N s For each batch of samples, perform the following steps:

[0109] Each batch of data is again subjected to one or more data enhancement processes, including randomly adding white noise, randomly flipping along the Doppler dimension, and randomly shifting a small amount along the distance dimension.

[0110] The enhanced data is sent to CNN for forward calculation to obtain three types of confidence: no one exists, adult exists, and child exists, and the corresponding loss CE is obtained based on the loss function t ;

[0111] Perform backpropagation on the CNN network model and update the parameters of each layer of CNN based on the gradient descent optimization algorithm;

[0112] In all After the above operations, use the updated CNN to Perform forward calculation on all samples of Loss of CE v , if CE v In the set N es If the value fails to decrease in the next cycle, the loop will be terminated early;

[0113] Get CE v The CNN model with the smallest cycle is output as the final model.

[0114] Among them, during the training process, Adam Optimizer can be used as the gradient descent optimizer, and cross entropy can be selected as the loss function. However, when the network output is normalized, sigmoid is first used for normalization operation before cross entropy is used for loss calculation; the learning rate is set to 0.001, the weight decay is set to 0.0001, and N s Set to 128, N es Set to 5.

[0115] Of course, the above is only an example of a deep learning model. In other embodiments, models with other structures or CNNs with different parameters, or different training parameters or training methods can be used, which will not be described in detail here.

[0116] It should be noted that the aforementioned embodiments are only examples. In some embodiments, the output and label assignment may not use one-hot encoding but may take other forms. For example, if only a scene with no one in the car or only one person is considered, Li = 1, 2, ..., 10 is directly generated as a label, and the position where the maximum value of the generated yi is found is used as the detection and recognition result, etc., which will not be described in detail here.

[0117] It's also understandable that radar coverage is relatively large, but users may not always need to detect signals within its coverage area. Therefore, in some embodiments, feature vectors are used to characterize the energy characteristics of the echo signal at different range bins within a preset distance range and / or preset Doppler range. For example, during beamforming, only data from a subset of Doppler bins and range bins is used, such as Doppler bins 0-32 and 96-127, and range bins 10-42. This retains only the portion of interest to the user, reducing computational workload, memory usage, and other resource usage, thereby improving detection efficiency.

[0118] To facilitate those skilled in the art to better understand the application of the target detection method provided in the above embodiment in the radar target detection process, the following will be described with reference to the application of the embodiment shown in Figures 3 to 5 as an example.

[0119] For the embodiment shown in FIG3 , after being applied to the complete target detection process, it will be as shown in FIG10 , that is, based on the process shown in FIG1 , after the inter-frame FFT (Frame FFT) processing, two-dimensional digital beamforming (2D-DBF) processing and DL-based target classification processing (DL-based Classification) operation are directly performed. That is, after the inter-frame FFT, CFAR processing is not performed, but 2D-DBF operation is performed on each region of interest (or unit interval) in the azimuth and elevation dimensions to obtain a number of RD maps corresponding to the region of interest; the above-mentioned RD maps are subsequently input into the constructed deep learning model (DL) for target judgment. In some optional embodiments, the relationship between the above-mentioned region of interest and the RD map can also be one-to-many, that is, at least two RD maps can be obtained based on one region of interest, and the specific number can be adjusted according to actual needs.

[0120] The DL input can be linear or dB-domain amplitude or power, and can also be normalized. The deep learning model constructed above can also classify the input, and the categories distinguished can include: whether there is a target, the attributes of the target, and the specific unit interval location (i.e., regional location) of each target.

[0121] For example, in an application scenario where the rear area of ​​a car is the area of ​​interest or the target area (i.e., the scenario shown in FIG6 ), and is divided into three seating areas (interval units) and corresponding three aisle areas, after the inter-frame FFT, a 2D-DBF in azimuth and elevation can be performed on the center positions of the aforementioned six areas of interest (the areas of interest are also set to three seating areas) to obtain six RD maps (or three RD maps) corresponding to the areas of interest. At least a portion of the aforementioned six RD maps (or three RD maps) is input into the constructed deep learning model for operations such as target judgment and differentiation. For example, the constructed deep learning model performs classification based on the input to determine whether there is a living target in the aforementioned area of ​​interest. If there is a living target, it can further determine whether the living target is an adult, child, infant, or pet, etc., and can further determine specific location information such as which seat or aisle area the living target is in.

[0122] In the process shown in Figure 10, the target detection method is a processing process based on dual-drive of model data. It has low requirements for signal processing and can effectively avoid steps such as the selection of signal processing parameters, regional parameters, and the design of CFAR logic and regional judgment logic, thereby greatly reducing the difficulty of solution implementation and design.

[0123] The embodiment shown in Figure 4, when applied to a complete target detection process, will result in the process shown in Figure 11. As shown in Figure 11, based on the process shown in Figure 1, after performing a range-dimensional Fourier transform (also known as a 1D-FFT), a 2D-DBF is performed, followed by frame data accumulation and inter-frame FFT processing. Specifically, before the inter-frame FFT, a 2D-DBF is performed on the center of the region of interest (e.g., a three-seat area) in terms of azimuth and elevation. This embodiment enables parallel processing of data and transmission time, effectively reducing processing time.

[0124] For the embodiment shown in FIG5, after being applied to the complete target detection process, it will be as shown in FIG12. Among them, as shown in FIG12, on the basis of the process shown in FIG1, that is, after the range-dimensional Fourier transform (Range FFT, also called 1D-FFT), the frame data accumulation can be completed after continuing 2D-DBF, and the DL-based target classification operation (Complex DL-based Classification) can be directly performed. That is, in the entire target detection process, no inter-frame FFT processing operation is performed. After 1D-FFT, 2D-DBF is directly performed on the center position of the area of ​​interest (such as the 3 seat area) to obtain the corresponding number of distance-frame maps (such as 3 range-frames). Subsequently, the above-mentioned distance-frame maps are input into the constructed (for example, complex) deep learning model to perform operations such as target differentiation and judgment.

[0125] In the embodiment shown in FIG12 , since the FFT processing step between frames is avoided, the amount of calculation can be effectively reduced; at the same time, if a complex deep learning model is adopted, a higher processing gain can be obtained compared to FFT, thereby achieving the purpose of improving target detection performance; in addition, this embodiment also effectively shortens the processing level, which can facilitate its efficient scheduling operations, etc.

[0126] As can be seen from the above, in the processes shown in Figures 4 and 6, the combination of 2D-FFT and DL-based classification can effectively improve the accuracy of target recognition. At the same time, the 2D-FFT steps and DL-based classification steps can be flexibly set between various steps in signal processing according to actual needs. For example, the 2D-FFT step can be set after the Range FFT or after the Frame FFT.

[0127] It should be noted that in the above embodiments, all FFT processing can be windowed FFT, and can also be implemented by SVA combined with FFT; alternatively, multi-frame sliding window FFT can replace other time-frequency transform processing, such as short-time Fourier transform, fractional Fourier transform, etc., which can also achieve effective detection of the target of interest; alternatively, multi-frame sliding window FFT can be replaced by high-pass FIR, Comb FIR or a filter optimized and synthesized according to an optimization function, so as to achieve the purpose of obtaining similar or even higher performance of multi-frame FFT processing with fewer hardware resources; alternatively, DBF can be replaced by Capon, MUSIC, ESPRINT and other algorithms and their variants, and the beamforming at the center of the area of ​​interest can be optimized and synthesized and / or the antenna arrangement can be optimized and synthesized to further improve the system target detection performance.

[0128] It should also be noted that the embodiments shown in Figures 10 to 12 are only solutions for target detection combined with multi-frame accumulation technology, but this does not mean that multi-frame accumulation is combined. For example, in the embodiment shown in Figure 12, inter-frame accumulation is omitted, and of course, etc., which will not be described one by one here.

[0129] As shown in Figures 13 to 21, the embodiment of the present application also provides another method for target detection. By adopting inter-frame accumulation combined with deep learning, coherent accumulation is performed on the frequency of human breathing, the signal-to-noise ratio of human targets and static strong clutter is improved, and feature data of empty vehicles (with or without interference), scenes with adults, and scenes with children are effectively constructed; at the same time, combined with a deep learning model, deep information mining is performed on the constructed feature data, thereby achieving effective detection of whether there are people in the car and accurate judgment of adults and children, and more accurate detection and identification of targets in spatial areas such as car cabins, indoors, and factories. Compared with point cloud information based on signal processing, the method proposed in this embodiment uses data with higher feature dimensions and richer information, which can achieve more accurate detection and identification of targets in the cabin. Specifically:

[0130] As shown in FIG13 , a method for target detection may include:

[0131] Step S11: For each chirp data in each frame, for example, DC filtering can be performed on the downlink ADC data in the BB. Optionally, the data collected by each RX channel for each chirp can be averaged along the fast time dimension, and the DC component can be subtracted from all sampling points of each RX channel. Then, the DC-filtered data can be windowed and then subjected to a 1D-FFT to obtain distance dimension information. The 1D-FFT data can then be moved from the BB to the CPU SRAM cache via DMA. Finally, the above operations are repeated for different chirps.

[0132] Step S12: When the number of frames of data cached in the CPU SRAM reaches a certain number, such as 128 frames, the 128 frames of data are moved from the CPU SRAM to the BB via DMA, and the following steps are performed:

[0133] First, DC filtering is performed on multiple frames of 1D data: the complex average along the frame dimension is calculated for these 128 frames of data, the average value of different channels and different distances is obtained as its DC component, and its DC component is subtracted from each distance unit of each channel.

[0134] Secondly, the DC-filtered data is windowed along the frame dimension, and then a 2D-FFT calculation is performed along the frame dimension to obtain the range-Doppler complex value x(c, r, v) of each transmitting and receiving channel.

[0135] After that, some Doppler units and range units are intercepted, such as 0-32 and 96-127 Doppler units and 10-42 range units. The energy of the complex value x(c, r, v) of each range unit and each Doppler unit of each channel is calculated, and then the logarithm is taken and normalized. The mathematical form is as follows:

[0136] Where P(c, r, v) is the normalized “energy” of virtual channel c, range unit r, and Doppler unit v, b0 and a0 are preset normalization parameters, and P max and P min The upper and lower limits of P(c, r, v) are preset. Generally speaking, the same b0, a0, and P are selected for different c, r, and v. max and P min Value (possible variations can be different c and different v and r partitions to select different b0, a0, P max and P min In one possible embodiment, b0 can be set to all possible log2(|x(c,r,v)| 2 ) mean or median, a0 is the value of all possible log2(|x(c,r,v)| 2 ) is obtained by dividing the difference between the 95% quantile and the 5% quantile by 256. This normalizes P(c,r,v) to between -127 and 127.

[0137] Next, P(c, r, v) is rearranged as shown in Figure 7: continuous in physical memory along the v dimension, followed by the r dimension, and finally the c dimension. In the figure, v0 and vNv represent the 0th and Nvth Doppler bins (for a total of Nv+1 Doppler bins), r0 and rNr represent the 0th and Nrth range bins (for a total of Nr+1 range bins), and c0 represents the 0th virtual channel, for a total of Nc+1 virtual channels. This (Nv+1)×(Nr+1)×(Nc+1) array is the input feature data for the deep learning model.

[0138] Finally, the above data is fed into the constructed deep learning model to determine whether there is someone in the cabin and whether it is an adult or a child, and the results are output.

[0139]

[0140] It should be noted that the deep learning model can be either CNN or Transformer. An embodiment of the model construction process is shown in Figure 14, and the main steps are:

[0141] Step S21: Obtain input feature data in the same way as above, and accumulate multiple processing data of multiple groups of experiments to construct input feature data set Each element P i is an (Nv+1)×(Nr+1)×(Nc+1) array, defined the same as in step 2.c) above.

[0142] Step S22, according to the experimental conditions Marking, that is, P i Generate the corresponding label L i , obtain the labeled input feature dataset

[0143] Step S23: The enhanced dataset is obtained by performing data enhancement by randomly adding white noise, randomly flipping along the Doppler dimension, and randomly shifting along the distance dimension. Note that the number of samples in the augmented dataset is much larger than that in the original dataset. For example, There are 5,000 samples of hollow cars, children, and adults, and 10 rounds of data augmentation can be performed to obtain an augmented dataset of 50,000 samples of each type. (You can choose to include or not include the original dataset ).

[0144] Step S24, Random sorting, taking α=0.3 ratio As a validation dataset The remaining data is used as the training dataset

[0145] Step S25, based on and Train deep learning models.

[0146] Optionally, the main steps of an embodiment are:

[0147] First, an uninitialized CNN model is constructed and the parameters of each layer are randomly initialized.

[0148] Secondly, loop the following steps 20 times (each time is called an epoch): Randomly divide into N b batches, each batch is N s The following steps are performed on each batch: first, the data of each batch is subjected to one or more data enhancement processes, including random addition of white noise, random flipping along the Doppler dimension, and random small translation along the distance dimension; then, the enhanced data is sent to CNN for forward calculation to obtain three types of confidence: no people, adult presence, and child presence, and the corresponding loss CE is obtained based on the loss function. t ,; Finally, the CNN network model is back-propagated and the parameters of each layer of CNN are updated based on the gradient descent optimization algorithm. After the above operations, use the updated CNN to Perform forward calculation on all samples of Loss of CE v , if CE v In the set N es If the value fails to decrease in the next cycle, the loop will be terminated early.

[0149] Then, take CE v The CNN model with the smallest cycle is output as the final model.

[0150] It should be noted that, in the process of model training, a possible embodiment can use Adam Optimizer as the gradient descent optimizer, select cross entropy as the loss function, set the learning rate to 0.001, set the weight decay to 0.0001, and N s Set to 128, N es Set to 5

[0151] Figure 15 shows a possible CNN architecture. A CNN with three layers of convolutional kernels is shown here, with the type, input, and output sizes of each layer also indicated. The input feature dimensions are 16×64×20, corresponding to 16 virtual channels, 64 Doppler units, and 20 range units. Each convolution operation consists of a two-dimensional convolution layer, a batch normalization layer, a ReLu nonlinear activation layer, and a max pooling layer. After three convolution operations, the resulting 128×2 feature vector is flattened to a one-dimensional 256×1 feature vector. After dropout, it passes through a 256×3 fully connected layer to obtain confidence scores for the three categories: empty vehicle, child present, and adult present. This confidence score is calculated using softmax to obtain the maximum value, which is used to classify the input as either empty vehicle, child present, or adult present.

[0152] It should be noted that:

[0153] 1. The method in this embodiment is different from the slow time processing between chirps in existing radar signal processing. It accumulates multiple frames of data and performs FFT processing between frames in the form of a sliding window.

[0154] 2. The method in this embodiment proposes a process for detection and classification using inter-frame FFT processing and CNN, without the need for CFAR and DoA of traditional signal processing to obtain point clouds for processing.

[0155] 3. The method in this embodiment details the innovative CNN input feature construction. Noteworthy points and possible variations include:

[0156] ① When calculating P(c,r,v), the log2 operation can be skipped or log 10 or ln operation, etc.

[0157] ② To calculate P(c, r, v), the complex data can be transferred to the MCU Mem via DMA and then performed in the MCU. Alternatively, the data can be implemented using the hardware engine in the BB and then directly transferred to the MCU Mem via DMA. Alternatively, part of the calculation can be implemented in the BB and then transferred to the MCU Mem via DMA and the remaining calculation can be performed in the MCU (e.g., modulo square and log2 operations can be implemented in the BB, while subtraction, division, and saturation truncation can be implemented in the MCU).

[0158] ③ P(c, r, v) can be quantized using double-precision floating point or single-precision floating point, or it can be quantized in int32, int16, int8 or uint32, uint16, uint8.

[0159] ④ The above data arrangement can be continuous along the r dimension, followed by the v dimension and the c dimension (as shown in Figure 23); it can also be the vcr or cvr dimension.

[0160] ⑤ In the deep learning model, you can choose the virtual channel as the channel of the model, or you can flatten the virtual channel dimension and use a single-channel deep learning model input. For example, the input of the deep learning model is 1×64×320, which contains 64 Doppler units, and the 20 distance units of 16 channels are flattened into 320.

[0161] ⑥ Only some channels can be selected to avoid the impact of channels with high noise or poor RF analog characteristics on the processing effect. For example, 13 channels out of 16 virtual channels (4 transmit and 4 receive) are used to construct 13×64×20 CNN input feature data to judge empty vehicles, children, and adults.

[0162] ⑦ The data after the inter-frame FFT can be truncated to save memory and reduce the amount of calculation, or it can be not truncated to achieve the best performance. If truncated, it can be performed at any time or a combination of the time after 1D-FFT, after 2D-FFT, or when calculating P.

[0163] As shown in Figure 16, the target detection method in the embodiment of the present application can also adopt the processing flow shown in Figure 16, that is, the data after the inter-frame FFT is incoherently accumulated by channel to obtain an array and the data is subjected to operations similar to the above-mentioned steps of intercepting part of the Doppler unit and the distance unit, and then sent as input feature data to the deep learning model for processing to obtain the classification result; the corresponding input channel number of the deep learning model is selected as 1.

[0164] As shown in Figure 17, the target detection method in the embodiment of the present application can also adopt the processing flow shown in Figure 17, and the data after inter-frame FFT is incoherently accumulated by channel to obtain two-dimensional data, and then only the energy of the points that exceed the set threshold is selected through CFAR, and a similar operation of intercepting part of the Doppler unit and distance unit is used as the input feature data, and the distance units and Doppler units that do not exceed the threshold are filled in, and then the input feature data is sent to the deep learning model for processing to obtain the classification result; the corresponding input channel number of the deep learning model is selected as 1.

[0165] It should be noted that a combination of the two embodiments of Figures 16 and 17 above can be adopted, that is, the data after inter-frame FFT is CFAR-ed by channel, and for each channel, only the energy of the points that exceed the set threshold is selected, and a similar operation to the above-mentioned step of intercepting part of the Doppler unit and the distance unit is used as the input feature data, and the distance units and Doppler units that do not exceed the threshold are filled in. Subsequently, the data of all channels are arranged in a similar manner to the above-mentioned rearrangement of P(c, r, v), and the input feature data is sent to the deep learning model for processing to obtain the classification result; the number of input channels of the deep learning model should be selected as the number of virtual channels used.

[0166] The deep learning model in the embodiments of this application can choose other CNNs or Transformers, as long as their input is adapted. As shown in Figures 18-19, a CNN model modified based on ResNet18 and the expansion of the Sequential module therein can be used. The output here can also be empty and non-empty.

[0167] The two figures in Figure 20 show examples of CNN input feature maps for adults and children, respectively. The left figure shows the input features for adults, and the right figure shows the input features for children. Each sub-figure corresponds to data from a different channel. The horizontal axis is Doppler units, and the vertical axis is range units. In this example, 20 range units and 64 Doppler units are captured.

[0168] Figure 21 shows how the model loss and classification accuracy change with the number of training epochs. It can be seen that the model converges after about 7 epochs and reaches an accuracy of 98%.

[0169] Figure 22 shows the confidence scores for adult and child samples. The horizontal axis represents the sample number, and the vertical axis represents the confidence score (i.e., score). In the experiment, the first 150 samples were samples of adults in the car, and the next 150 samples were samples of children in the car. For each sample, a possible deep learning model calculated its confidence score for adults and children, generating two blue and orange points for each sample. When the confidence score for an adult is greater than that for a child, the person in the car is determined to be an adult; when the confidence score for a child is greater than that for an adult, the person in the car is determined to be a child. As shown in the figure, the trained CNN model can correctly determine whether the person in the car is an adult or a child.

[0170] In some embodiments, the target detection method, as shown in FIG24 , includes:

[0171] Step 10: Perform range-dimensional FFT processing on the echo signal to obtain 1D-FFT data.

[0172] Step 20, performing digital beam synthesis on the 1D-FFT data to obtain a preset number of frame data; and

[0173] Step 30: Perform target classification processing on the preset number of frame data based on a machine learning model to achieve target detection.

[0174] The above detection of the target in the region of interest includes at least one of the following operations: judgment, positioning, and identification.

[0175] By directly performing beamforming on a predetermined area after performing a range-dimensional FFT, and then using multiple frames of beamformed data to directly perform target classification processing based on a machine learning model, this method effectively detects the presence of occupants in the vehicle and accurately identifies adults and children. Compared to point cloud information based on traditional signal processing, this embodiment's method places lower demands on processing units (such as BB units) and utilizes data with higher feature dimensions and richer information, enabling more accurate in-cabin target detection and recognition, enabling applications such as CPD and SBR.

[0176] In an exemplary embodiment, the machine learning model may be a complex machine learning model, such as a complex neural network model.

[0177] It should be noted that the main difference between the embodiment shown in FIG24 and the aforementioned embodiment is that the model used in the embodiment shown in FIG24 is a machine learning model, while the model used in the aforementioned embodiment is a deep learning model. In order to adapt to the input of the complex machine learning model, in some embodiments, the feature vector can be converted. The conversion method is to convert the power and phase into the real part and the imaginary part, that is:

[0178] z real (r,v,b)=Re[P norm (r,v,b)·exp(1j·φ(r,v,b))];

[0179] z img (r,v,b)=Im[P norm (r,v,b)·exp(1j·φ(r,v,b))];

[0180] z(r,v,b)=z real (r,v,b)+1j·z img (r,v,b);

[0181] Where z real and z imgare the normalized real and imaginary parts, respectively. φ(r, v, b) is the phase of chirp v at range cell r of beam b. z(r, v, b) is the normalized complex signal. Furthermore, in some embodiments, z(r, v, b) may be expressed in double-precision floating point or single-precision floating point, or may be quantized in int32, int16, int8, uint32, uint16, or uint8, or in other quantization formats.

[0182] In addition, different loss functions can be used based on different models. For example, the complex neural network model can remove the absolute value (abs) layer of the last layer in the complex-valued CNN. At this time, the complex-valued CNN output is a complex output. At this time, the following loss function can be used:

[0183] Loss=CE(Re(y),t) / 2+CE(Im(y),t) / 2

[0184] Among them, y is the forward calculation output of Complex-valued CNN for a sample, which is a vector with the length of the classification number. i is the i-th scalar in y; t is the one-hot encoding label of the sample, which is also a vector with the length of the classification number, t i is the i-th scalar in t. Other hyperparameters such as learning rate and weight decay can have other values.

[0185]

[0186] The embodiment shown in FIG24 is basically the same as the previous embodiments in other features or implementation methods of the features, and will not be described in detail here.

[0187] Of course, in the above embodiments, the model is used to replace the corresponding digital signal processing process, for example, replacing the angle estimation, CFAR, etc. In some embodiments, the model can also be used to further process the target obtained by digital signal processing.

[0188] Based on this, in some embodiments, as shown in FIG25 , the target detection method includes the following steps:

[0189] Step 30: After performing distance-dimensional FFT processing based on the echo signal, 1D-FFT data is obtained, and multi-frame joint processing is performed on the 1D-FFT data to obtain an RD spectrum.

[0190] Step 40: obtaining target point cloud data based on the RD map, and performing target classification processing based on machine learning on the target point cloud data to detect the target in the region of interest.

[0191] The above detection of the target in the region of interest includes at least one of the following operations: judgment, positioning, and identification.

[0192] The target detection method and related device provided in this embodiment utilize multi-frame joint processing technology, namely, inter-frame accumulation, to implement target detection solutions within enclosed spaces such as cabins, indoor spaces, and factory buildings. This effectively improves detection rates, reduces false alarms, and significantly enhances the accuracy of angle estimation. This enables precise detection of special or weak targets within cabins, such as infants and young children, enabling applications such as Child Presence Detection (CPD) and Safety Belt Reminder (SBR). Furthermore, through machine learning-based target classification processing of point cloud data, more refined regional determination and more accurate identification of targets within the region can be achieved. For example, this allows for more accurate determination of the presence and location of personnel within enclosed environments such as cabins.

[0193] In an exemplary embodiment, step 30 includes: performing range-dimensional FFT processing on the echo signal within the chirp to obtain 1D-FFT data; accumulating frame data on the 1D-FFT data until a preset amount of data is accumulated, and then performing inter-frame FFT processing to obtain an RD spectrum. For example, a sliding window approach can be used to read a preset number of frames of 1D-FFT data and perform inter-frame FFT processing. This means that accurate detection of targets within the cabin can be achieved using multi-frame joint processing technology. This multi-frame joint processing approach can involve performing sliding window FFT processing on multiple frames of data to obtain a range-Doppler map, or employing FIR (Finite Impulse Response) or other complex time-frequency transforms before performing CFAR, DOA, and other processing operations on the region of interest. Furthermore, this approach can be combined with pre-defined region judgment logic and parameters to achieve detection and location of living targets such as adults, children, and pets, as well as other non-living targets. CFAR can be NR-CFAR, RD-CFAR, or DAE (Doppler-Azimuth-Elevation)-CFAR in the Doppler dimension. CFAR and DoA can be post-processed using clustering, false alarm suppression, and multi-process point cloud association. Regional parameters can be determined by performing at least one of clustering operations on the point cloud detected after a sliding window multi-frame process, or performing outlier detection and elimination on the point cloud detected after a sliding window multi-frame process, or a combination of at least two of these operations.

[0194] In an exemplary embodiment, the step of obtaining target point cloud data in step 40 includes: performing incoherent accumulation of the RD spectra of each channel, performing constant false alarm processing based on noise estimation based on the incoherent accumulation results to obtain candidate target detection points, estimating the azimuth median angle and elevation angle of the candidate target detection points, and obtaining target point cloud data based on the azimuth median angle and elevation angle. By performing incoherent accumulation of the RD spectra of each channel, the signal-to-clutter ratio (SCR) of the human target and strong static clutter can be effectively improved.

[0195] In some embodiments, the constant false alarm processing includes: performing incoherent accumulation on the RD spectrum of each channel, estimating the noise floor of each distance unit after the incoherent accumulation, obtaining a noise floor estimation value of each distance unit, and performing incoherent constant false alarm processing using the noise floor estimation value, wherein, when estimating the noise floor of each distance unit after the incoherent accumulation, the noise floor of each distance unit is corrected using the global noise floor.

[0196] In some embodiments, the correction method includes: correcting the noise floor of each distance unit in the following manner: n′ i =min(n i ,α*n g ), where ni is the original noise floor estimate of the i-th distance unit, and n g is the mean of the noise floor estimates of multiple range cells, n′ i is the corrected noise floor estimate for the i-th range cell, and α>1. By correcting the noise floor, we can avoid the target being undetected due to an excessively high noise floor.

[0197] In an exemplary embodiment, the step of performing machine learning-based target classification processing on the target point cloud data in step 40 to detect (e.g., determine) the target in the region of interest includes: inputting the target point cloud data into a pre-trained first machine learning classifier to obtain a first detection result, wherein the first detection result indicates whether the candidate target detection point is a valid candidate target detection point, and the target point cloud data includes coordinate data. The coordinate data refers to coordinates in a rectangular coordinate system, including x-coordinates, y-coordinates, and z-coordinates. In some embodiments, the data input into the first machine learning classifier also includes one or more of the following: distance data, signal-to-noise ratio, amplitude value, azimuth median angle, and pitch angle.

[0198] In an exemplary embodiment, the target point cloud data is subjected to machine learning-based target classification processing in step 40 to detect (e.g., locate and identify) targets in a region of interest. The processing includes: processing the target point cloud data to extract region information of candidate target detection points; inputting the region information into a pre-trained second machine learning classifier to obtain a second detection result, wherein the second detection result indicates whether a target exists in the predetermined region. In some embodiments, the region information of the candidate target detection points includes one or more of the following information: the number of valid candidate target detection points in each predetermined region; the proportion of valid candidate target detection points in each predetermined region to the total number of valid candidate target detection points; and the mean and variance of the distance, azimuth median angle, elevation angle, and constant false alarm signal-to-noise ratio (CFAR) of all valid candidate target detection points in each predetermined region.

[0199] Combining classification and / or recognition with a machine learning classifier can achieve finer area division than the preset logic to accurately classify the seats corresponding to the radar-processed point cloud.

[0200] The first machine learning classifier or the second machine learning classifier can be any one or more of the following classifiers: support vector machine, random forest, decision tree, mixed Gaussian model, KNN, hidden Markov, and multi-layer perceptron.

[0201] When applied to enclosed or relatively enclosed spaces, the system can be pre-divided to define and detect different areas of interest (divided areas). For example, when detecting targets within a vehicle cabin, the radar monitoring area can be divided into seating areas and aisle areas, for example. Parameter types and thresholds can then be preset for each type of area, and the corresponding processing steps can be combined to achieve precise detection of targets of interest within a specific area.

[0202] In some embodiments, for family cars, the interior space can generally be simply divided into headroom, rear space, and trunk space. If the rear space is the key monitoring area, it can be further divided into seat zones and aisle zones. The seat zones can be divided into a corresponding number of seat zones based on the seats. Similarly, the aisle zones can be divided into a corresponding number of aisle zones corresponding to the seat zones. For example, for a five-seater family sedan, the three seats in the rear space can be divided into three seat zones and three aisle zones. For different types of zones (or zones or zones), corresponding parameter types and thresholds can be preset, and adaptive signal data processing methods and steps can be used to achieve accurate detection of targets of interest (specific targets) within specific zones or zones. Adjacent zones can have partial overlap, or they can be adjacent or separated by a gap of a preset width.

[0203] The following is a detailed description of the technical solution of the present disclosure, based on the technical concept of frame-level Fourier transform, with reference to the accompanying drawings. Specifically, after performing a fast-time Fourier transform on the chirp in the echo signal, at least two frames of fast-time Fourier transform data are accumulated. Then, a frame-dimensional Fourier transform is performed on the at least two frames of data to obtain a range-Doppler (RD) spectrum, and target range and / or velocity estimation is performed based on the RD spectrum.

[0204] As shown in FIG26A , after the echo signal undergoes operations such as ADC (analog-to-digital) conversion and sampling, range DC removal and range Fourier transform (also known as 1D-FFT or fast time FFT) are sequentially performed, followed by multi-frame accumulation (e.g., storing 128 frames of data), Doppler DC removal, frame FFT, and non-coherent integration. Constant false alarm detection (e.g., peak-based CFAR (detection with peak selection)) is then performed based on the noise obtained by noise estimation (e.g., using a noise variance estimator). Finally, angle detection and target classification are performed to obtain target range, velocity, and / or angle information. For example, azimuth and elevation estimation can be performed based on the CFAR results, specifically by using a digital beamformation (DBF) (DBF). It is implemented using technologies such as digital beamforming (forming) and DoA (direction of arrival estimation).

[0205] After the azimuth / elevation angle estimation is performed, the output target point cloud data can be processed in combination with a machine learning model to accurately detect targets in the area of ​​interest. For example, for the single target point cloud data obtained after the azimuth / elevation angle estimation, the false alarm suppression technology of ML (Machine Learning, such as SVM or RF algorithm) can be combined to suppress non-ideal data such as false alarms generated by noise, stationary clutter or target multipath. That is, for the target point cloud data such as the pitch DBF&DoA output, one or more of the following operations are continued: ML-based false-alarm suppression, clustering, and ML-based target classification. The clustering process can be agglomerative clustering or DBSCAN algorithms. In an exemplary embodiment, the point cloud data after clustering processing can also be input into a trained machine learning model (such as SVM or RF) to determine whether there is a person in the current processing. If there is, the person's location can be further determined, and operations such as distinguishing and judging physical targets such as adults, children, and infants can be achieved.

[0206] In the embodiment shown in FIG26A , the influence of non-ideal factors such as noise, stationary clutter and target multipath can be effectively reduced, and the difficulty of selecting regional parameters and the difficulty of designing complex regional judgment logic can be effectively reduced, thereby more effectively utilizing point cloud information, especially for applications in complex scenarios, and achieving better target detection and discrimination performance.

[0207] In some optional embodiments, for an electromagnetic wave sensor having a BB (Baseband) unit and an MCU (Microcontroller Unit) unit, based on the flowchart of target detection shown in Figure 26A, operations such as frame data accumulation (store 128 frames) and target classification (for example, classification can be performed based on a region histogram (Region HIST)) can be performed in the MCU module, while range DC removal (Range DC removal) and Doppler DC removal (Doppler DC removal) can be performed in the BB module or the MCU module. In the process shown in Figure 26A, DC filtering of downlink ADC data in the BB module is taken as an example for explanation.

[0208] As shown in Figure 26A, after performing 1D-FFT processing or obtaining distance dimension information in the BB module, the 1D-FFT data can be cached in the MCU module. After caching to a preset data amount (such as 128, 56, or 32 frames), the data cached to the preset number of frames can be DC removed and frame-level FFT (frame-level FFT) can be performed in the frame dimension to obtain an RD (Range-Doppler) diagram.

[0209] When the system includes multiple channels, as shown in FIG26A , non-coherent integration can be performed on the RD spectra of the multiple channels, and the noise floor of each range bin in the accumulated data can be estimated using the NVE (noise variance estimator) module. For example, the noise floor estimation can be performed based on a preset formula, and subsequent NR (Noise Reference)-CFAR processing can be performed based on the value of the noise floor estimation. The above preset formula can be: n i =min(n i ,α*n g ), n i is the noise floor estimate of the i-th range bin, n gis the noise floor estimate of the last range bin of interest. α is a coefficient, which is a number greater than 1 and can be set and updated based on requirements, engineering data, experience, etc.

[0210] For angle estimation, as shown in FIG26A , azimuth dimension DBF and DoA, as well as elevation dimension DBF and DoA, may be performed on the CFAR detection point data to obtain azimuth and elevation angle estimation data of each CFAR detection point.

[0211] Figure 26B is a flowchart of another embodiment of the present application for implementing target detection based on frame-level FFT. The difference from Figure 2A is that the subsequent classification processing based on machine learning is slightly different. For the target point cloud data output by pitch DBF&DoA, one or more of the following operations are performed: ML-based target point classification processing (ML-based points classification), region feature extraction (Region feature extraction), ML-based target region classification processing (ML-based region classification), wherein the ML-based target point classification processing is used to realize the judgment of the target in the region of interest, that is, to classify the target point cloud data and obtain valid target detection points (that is, the aforementioned valid candidate target detection points) therefrom; region feature extraction can assist in classification, for example, clustering can enhance the classification performance, which helps to improve the accuracy and performance of classification; the ML-based target region classification processing is used to realize the positioning and identification of the target in the region of interest, that is, to determine whether there is a target in the preset area.

[0212] In the above-mentioned embodiment, the target's speed is determined by using a frame-level FFT instead of the traditional chirp-level Doppler FFT to accurately detect the target of interest. The frequency of updating the results can be further increased by performing sliding window FFT processing on multiple frames, thereby improving the real-time performance of the system. In addition, the amount of data to be processed can be reduced by processing only the distance and / or Doppler region of interest. In addition, when estimating background noise, the background noise is obtained by taking the minimum value based on the estimate of the current range bin combined with the estimate of the last range bin of interest, which is more suitable for application scenarios in relatively sealed environments such as vehicle cabins. When making regional logical judgments, by counting the target points detected in each frame according to the preset area design and judging the counting results according to preset rules, it is possible to accurately judge whether there is a target in each preset area.

[0213] It should be noted that, similar to the aforementioned embodiment, when performing target detection, there is no limitation on the implementation of the corresponding processing or its replacement. For example, the FFT processing can be windowed FFT, or SVA combined with FFT can be used for implementation, or multi-frame sliding window FFT can replace other time-frequency transform processing, such as short-time Fourier transform, fractional Fourier transform, etc., which can also achieve effective detection of the target of interest. They will not be described in detail here.

[0214] In some embodiments, after the inter-frame FFT (Frame FFT) processing, two-dimensional digital beamforming (2D-DBF) processing and ML-based target classification processing (ML-based Classification) operations can be directly performed, that is, after the inter-frame FFT, CFAR processing is not performed, but 2D-DBF operations are performed on the azimuth and elevation angles of each region of interest (or unit interval) to obtain a number of RD maps corresponding to the region of interest; the above-mentioned RD maps can then be input into a pre-built machine learning model (ML) for target judgment. In some optional embodiments, if the inter-frame FFT method is not used, but FIR, STFT (Short-Time Fourier Transform) and other methods are used for processing, the above-mentioned RD map will be a one-dimensional map output corresponding to FIR, and a three-dimensional map output corresponding to STFT time-frequency transform.

[0215] The embodiments of the present application can be mutually referenced and compatible when there is no conflict, and the order and configuration of each functional module can be adjusted as needed. For a system with a BB module and an MCU module, when the system detects a target by emitting electromagnetic waves and receiving corresponding echo signals, the various steps in the target detection methods of the embodiments of the present application can be configured to run in the BB module and / or the MCU module based on actual needs and the data processing capabilities and timeliness of the system operation. The relevant examples in the figure can serve as a reference for some of the options.

[0216] By adopting the method of the embodiment of the present application, through multi-frame joint processing technology, coherent accumulation is performed on the frequency of human breathing in the Doppler domain and on the multi-antenna channels of the preset orientation in the spatial domain, thereby effectively improving the signal-to-clutter ratio of human targets and static strong clutter; and the radar point cloud is clustered and classified in combination with the machine learning model to achieve more refined regional judgment than the preset logic, so as to accurately classify the seats corresponding to the radar processing point cloud. By improving the signal-to-noise ratio, unsupervised clustering self-labeling, and refined machine learning classification, it is possible to more accurately judge the presence and location of personnel in closed environments such as cabins. In addition, compared with rule-based regional judgment, the model construction and processing process reduces the design of rules, reduces the difficulty of parameter adjustment, and increases the convenience and applicability of the method.

[0217] Based on the above solution, after testing using real-world data, it was shown that for top-mounted radar installation scenarios, a high detection rate, as well as extremely low missed detection and false alarm rates, can be achieved. For example, when detecting targets within a cabin, a detection rate of over 99% can be achieved, a missed detection rate of less than 0.5%, and a false alarm rate of less than 1%.

[0218] The present application also provides a method for detecting a target within a specific target area (e.g., a closed or semi-closed area). The method comprises: inputting target point cloud data into a pre-trained first machine learning classifier to obtain valid candidate target detection points; extracting region information for the valid candidate target detection points; and inputting the region information into a pre-trained second machine learning classifier to obtain target detection results. The specific steps are described in the preceding or following embodiments and are not further detailed here.

[0219] It should be noted that the implementation of the steps in the process shown in FIG26B is substantially the same as that in the embodiment shown in FIG1 , and the same parts will not be described in detail here.

[0220] In the process shown in FIG26B , when estimating the azimuth median angle of the candidate target detection point, for each candidate target detection point T i Perform the following operations: extract the 2D-FFT data corresponding to the detection point, select the azimuth antenna and perform azimuth DBF (digital beamforming) based on the azimuth steering vector to obtain the azimuth DBF spectrum. The DBF spectrum shows how the signal power or intensity in different directions changes with frequency. Estimate the azimuth intermediate angle θ based on the azimuth DBF spectrum. i,j The relationship between the azimuth median angle and the azimuth angle is: azimuth median angle = sin (azimuth angle) cos (pitch angle). When estimating the pitch angle of the candidate target detection point, for each candidate target point T i,j According to its azimuth intermediate angle θ i,j And the array arrangement generates its pitch direction steering vector sv(i,j) respectively, for each candidate target point T i,j Perform pitch-dimensional DBF to obtain its pitch-dimensional DBF spectrum, and estimate the pitch angle based on the pitch-dimensional DBF spectrum Among them, i represents the sequence number of the CFAR point (such as the range unit), j represents the sequence number of multiple candidate target points of a certain CFAR point at different azimuths, and k represents the sequence number of multiple candidate target points of a certain CFAR point at different elevations.

[0221] Furthermore, in some exemplary embodiments, when estimating the azimuth intermediate angle and the elevation angle, the azimuth intermediate angle and the elevation angle can be estimated directly by performing a two-dimensional DBF for the azimuth intermediate angle and the elevation angle, or other super-resolution algorithms such as the minimum variance unbiased estimation (Capon) algorithm and the multiple signal classification (MUSIC) algorithm can be used to estimate the intermediate angle and the elevation angle. Therefore, when extracting target information, all candidate target points T that meet the candidate target point detection conditions (e.g., exceed the CFAR threshold) can be selected. i,j,k Extract information, including but not limited to the range bin index of the target point, Doppler bin index, distance r i,j,k , Doppler frequency f i,j,k 、Azimuth intermediate angle θ i,j,k , pitch angle SNRsnr of CFAR i,j,k , the amplitude of each channel amp i,j,k,c etc., where c is the channel index. Then the coordinate system transformation is performed: for each candidate target point T i,j,k The distance r i,j,k 、Azimuth intermediate angle θ i,j,k , pitch angle Perform coordinate system conversion to obtain its position in the rectangular coordinate system: x, y and z. The converted data format is as follows:

[0222] x i,j,k =r i,j,k ·sin(θ i,j,k )

[0223] Thus, the pre-trained first machine learning classifier is used to classify the area to which the candidate target belongs, that is, the candidate target point T i,j,k As input, the first machine learning classifier is used to classify the region to which each candidate target point belongs, so as to determine the candidate target point T i,j,k Is it a valid target point or an invalid interference point in a certain area? A possible input example of the first machine learning classifier is: i,j,k 、x i,j,k 、z i,j,k 、y i,j,k 、snr i,j,k ,amp i,j,k,c , the machine learning classifier can be, for example, a support vector machine (SVM) or a random forest (RF).

[0224] It should be noted that the first machine learning classifier can have multiple forms, or even a fusion of multiple forms. For example, it can be a fusion of support vector machines, decision trees and mixed Gaussian models. A weight is set for the output result of each classifier to obtain a weighted average of all classifiers. The weighted average result is used to make another judgment to obtain the final judgment result.

[0225] The input of the first machine learning classifier is only an example. In other embodiments, the T of the selected candidate target point is i,j,k The attribute can be "r i,j,k 、x i,j,k 、z i,j,k 、y i,j,k 、f i,j,k 、snr i,j,k ,amp i,j,k,c " may include all or part of the coordinate data (the above-mentioned x, y, z values), or may include one or more of the distance data (the above-mentioned r), Doppler frequency (the above-mentioned f), signal-to-noise ratio (the above-mentioned snr), and amplitude value (the above-mentioned amp) in addition to the coordinate data. Alternatively, those skilled in the art may increase the input values ​​of the first machine learning classifier based on the concepts of the present disclosure. For example, the coordinate data may be replaced by the azimuth median angle and the pitch angle, or the azimuth median angle and the pitch angle data may be input into the first machine learning classifier together.

[0226] Then, the pre-trained second machine learning classifier is used to classify each region, that is, regional feature extraction is performed: the number of valid target points in each region and the proportion of valid target points to the total valid target points w are counted. l , and calculate the distance, azimuth median angle, pitch angle, and mean and variance of SNR of all valid target points in each area, which are the mean of distance Variance of distance σ r,l , the mean of the median angles of the azimuths Variance of the azimuth median angle σ θ,l , the mean of the pitch angle Variance of pitch angle Mean SNR SNR variance σ snr,l , where l is the index of the region; the above regional features The data is sent to the second machine learning classifier for classification to determine whether there is anyone in each area.

[0227] For example, the above statistics on the number of valid target points in each area and the proportion of valid target points to the total valid target points w lWhen calculating the total valid points, valid points that are determined to belong to multiple areas can be counted repeatedly (suitable for multi-label classification, such as a scene where people exist in both area A and area B), or they can be counted non-repeatedly (suitable for single-label classification). This disclosure does not impose any restrictions on this.

[0228] The second machine learning classifier can have multiple forms, or even a fusion of multiple forms. For example, it can be a fusion of a support vector machine, a decision tree, and a mixed Gaussian model. A weight is set for the output result of each classifier to obtain a weighted average of all classifiers. The weighted average result is used to make another judgment to obtain the final judgment result.

[0229] The input of the second machine learning classifier is only an example. In other embodiments, the T of the selected candidate target point is i,j,k The properties can be Alternatively, those skilled in the art can increase the input value of the second machine learning classifier based on the concept of the present disclosure.

[0230] The first machine learning classifier for judging the effective points can be any one of the following classifiers: SVM, RF, decision tree, mixed Gaussian model, K-nearest neighbor (KNN), hidden Markov, multi-layer perceptron, etc. The training steps of the classifier model include:

[0231] 1. Generate valid point labels. One possible process is:

[0232] The areas are divided in advance, as shown in an example in Figure 6, where the horizontal axis is the X direction (vehicle width direction) and the Z direction (vehicle tail-vehicle front direction). It divides the three seats in the back row and the three aisles in front of the three seats into six areas. These areas may overlap with each other or not; certain areas in the car may belong to multiple preset areas at the same time, or may not belong to any area. In other embodiments, when dividing the areas, the areas can be subdivided in combination with the target personnel. For example, the back seats A, B, and C can be divided into: adult seat A, adult seat B, adult seat C, child seat A, child seat B, and child seat C. In this way, when the application scenario requires determining whether there are people in the back rows A, B, and C and whether they are adults or children, the results can be obtained directly. It is only necessary to add corresponding classifications to the effective point determination and area determination, and conduct training.

[0233] For data from various scenarios, including scenarios with people in seat A, B, C, or no one interfering, the above steps 1-10 are used to obtain all candidate target points T in each scenario. i,j,k Attributes, such as optional: r i,j,k 、x i,j,k 、z i,j,k 、y i,j,j 、f i,j,k、snr i,j,k ,amp i,j,k,c In all or part of the , traverse various area scenes, such as the A, B, C seats or the A, B, C aisle areas in Figure 5, and perform the following operations:

[0234] 1.1 Perform cluster analysis on all candidate target point data of the scene and eliminate outliers. In some embodiments, a density-based clustering algorithm (DBSCAN) can be used to perform cluster analysis on these candidate target point data; in other embodiments, other clustering algorithms can be used to generate valid point labels, or manual labeling can be performed.

[0235] 1.2 The class with the most types after cluster analysis is selected as the true target in the area and its label is set to 1; all other candidate points are false targets and their labels are set to 0. In other embodiments, all non-noise candidate target points in the DBSCAN cluster analysis can be selected as true target points and their labels are set to 1, while the noise candidate target points in the DBSCAN cluster analysis are selected as invalid target points and their labels are set to 0; or a class with a large number of samples can be artificially selected as the true target in the area and its label is set to 1, while all other candidate points are false targets and their labels are set to 0.

[0236] For empty vehicle and interference scenarios, all candidate target points are set as invalid target points and their labels are set to 0.

[0237] 2. Model training is performed based on the generated labels. The main steps are: traverse various area scenes, such as seats A, B, and C or aisles A, B, and C, and perform the following operations:

[0238] 2.1 Read the attributes and labels of all candidate target points in the area, denoted as S{Xi} and S{Li} respectively, and read the attributes and labels of all candidate target points in the empty vehicle and interference scenes, denoted as S{X0} and S{L0} respectively; each element in S{Xi} is an N×1 vector. One possible implementation is to use the r i,j,k 、x i,j,k 、z i,j,k 、y i,j,k 、f i,j,k 、snr i,j,k ,amp i,j,k,c Any 6 scalars in the attribute (such as r i,j,k 、x i,j,k 、z i,j,k 、y i,j,k 、f i,j,k and snr i,j,k ) to identify the effective point, considering a total of 16 MIMO virtual channels, then S{X lEach element in} is a 22x1 feature vector;

[0239] 2.2 Combine the attributes and labels of these target points to obtain the training input data S{X}=S{Xi}∪S{X0} and the training labels S{L}=S{Li}∪S{L0};

[0240] 2.3 Train the machine learning classifier on the input data S{X} and label S{L} to obtain the classifier model M v,l .

[0241] For the application scenario of distinguishing the six areas of rear seats A, B, C and aisles A, B, C, after executing the above steps, six machine learning classifier models M can be obtained. v,l , where l = 1, 2, ..., 6 correspond to the six areas of the rear seats A, B, C and aisles A, B, C respectively. Each machine learning classifier model M v,l It can be based on the attributes of a candidate target point, such as r i,j,k 、x i,j,k 、z i,j,k 、y i,j,k 、f i,j,k 、snr i,j,k ,amp i,j,k,c The 22x1 feature vector composed of the candidate target point is used to determine whether the candidate target point is a valid target point in each area.

[0242] For example, the first machine learning classifier can use one classifier for multiple classifications, or it can use a classifier with a number less than the number of areas for hierarchical multi-classification. For example, to classify 6 seating areas, classifier 1 can be designed to realize the classification of seats and aisles, and then classifiers 2 and 3 can be designed to realize the classification of aisles ABC and the classification of seats ABC respectively.

[0243] Exemplarily, oversampling and / or undersampling operations may be performed when training the first machine learning classifier. Oversampling refers to balancing the quantitative differences between different categories in a data set by increasing the number of copies of minority class samples or synthesizing new samples. By adding data to minority class samples, the sample distribution between each category is made more balanced, thereby improving the model's learning ability for minority class samples. Undersampling refers to balancing an unbalanced data set by reducing the number of majority class samples so that the sample sizes between each category are closer. By deleting majority class samples, the number of samples in each category is made equivalent, thereby reducing the model's preference for the majority class.

[0244] When using the first machine learning classifier to determine valid points, a candidate target point may be determined to belong to multiple regions, such as the classifier model M v,1 and M v,2When the output of all classifiers is 1 and the output of other classifier models is 0, the candidate target point is determined to be a valid target point for seat A and seat B.

[0245] The second machine learning classifier for judging whether there are people in each area can also be any one of the classifiers such as SVM, RF, decision tree, mixed Gaussian model, KNN, hidden Markov, multi-layer perceptron, etc. The training steps of the classifier model include:

[0246] Traverse various area scenes, such as seats A, B, and C or aisles A, B, and C, and perform the following operations:

[0247] 1. Read the regional features of each frame sliding window processing of the scene generated by the above steps 1-11, such as As the input positive sample data, each element is a 9×1 feature vector, and all sample labels are set to 1;

[0248] 2. Read the regional features of each frame sliding window processing of other scenes generated by steps 1-11 above as input negative sample data, and set all sample labels to 0;

[0249] 3. Merge the positive and negative sample data and labels. When the number of positive samples is less than the number of negative samples, the positive samples can be resampled with a repetition factor of γ = N n / N p , where N n and N p are the number of negative samples and positive samples respectively;

[0250] 4. Train the second machine learning classifier to obtain the classifier model M c,l

[0251] For the application scenario of distinguishing the six areas of rear seats A, B, C and aisles A, B, C, after executing the above steps, six machine learning classifier models M can be obtained. c,l , where l = 1, 2, ..., 6 correspond to the six areas of the rear seats A, B, C and aisles A, B, C respectively. Each machine learning classifier model M c,l It can be determined whether there is anyone in area l. If the regional features of a certain sliding window processing are input, the second machine learning classifier model M c,l The output is 0, then it is determined that there is no one in the area l. If the machine learning classifier model M c,l If the output is 1, it is determined that there is a person in area 1. Note that based on this processing flow, it is possible to determine the presence of multiple people.

[0252] In an exemplary embodiment, an empty vehicle scene may be output as a separate category.

[0253] When using the second machine learning classifier to judge valid points, a candidate target point may be judged to belong to multiple regions, such as the classifier model M c,1 and M c,2 When the output of all classifier models is 1 and the output of other classifier models is 0, it is determined that there are people in both seat A and seat B in the scene; when all classifier models M c,l , when the outputs of l=1,2,...,6 are all 0, it is determined that there is no one in the scene.

[0254] Exemplarily, the second machine learning classifier may use one classifier for multi-classification, or may use a classifier with a number less than the number of regions for hierarchical multi-classification.

[0255] Exemplarily, oversampling and / or undersampling operations may be performed when training the second machine learning classifier. Oversampling refers to balancing the quantitative differences between different categories in the data set by increasing the number of copies of minority class samples or synthesizing new samples. By adding data to the minority class samples, the sample distribution between each category is made more balanced, thereby improving the model's learning ability for minority class samples. Undersampling refers to balancing the unbalanced data set by reducing the number of majority class samples so that the sample sizes between each category are closer. By deleting majority class samples, the number of samples in each category is made equivalent, thereby reducing the model's preference for the majority class.

[0256] Unlike the slow time processing between chirps used in radar signal processing in related technologies, the present embodiment accumulates multiple frames of data and performs inter-frame FFT processing in a sliding window format, improving the signal-to-clutter ratio between human targets and strong static clutter. The present embodiment utilizes a machine learning model for regional segmentation, which makes richer use of information than spatial segmentation and regional determination strategies based on preset rules. It can also be fully automated without human intervention, significantly reducing the burden of parameter adjustment.

[0257] The embodiments of the present application are not only applicable to human target detection and positioning scenarios in vehicles, but also to other similar application scenarios such as indoor personnel detection and factory personnel detection.

[0258] The above operations can be performed in the CPU or MCU, or in the baseband unit (BB), or partially by the CPU or MCU and partially by the BB. For example, 1D-FFT, 2D-FFT, CFAR, DBF, and DOA are performed in the BB, while target detection and recognition are performed in the MCU.

[0259] Figures 26A and 26B show the results of DBSCAN clustering for all points in the Seat C scenario across multiple experiments. The black points in the figures are candidate target points that DBSCAN determined to be noise, while the other colored points represent different classification clusters. According to the above example, the classification cluster with the most samples (as outlined in the black box in the figure) is selected as the valid point in the Seat C area, and the remaining points are invalid points in Seat C.

[0260] Figures 26C and 26D show the results of DBSCAN clustering and labeling all points from multiple experiments considering seats A, B, and C. The clusters with the most valid points for seats A, B, and C are shown in the black outlines in the figures.

[0261] Compared to existing technologies, the method in the embodiment of the present application adopts a new inter-frame accumulation method, performs coherent accumulation based on the frequency of human breathing, and improves the signal-to-noise ratio between human targets and static strong clutter; and uses a clustering method to eliminate abnormal outliers to achieve self-labeling of radar point cloud samples; combined with machine learning classifiers such as SVM, random forest or mixed Gaussian model, it achieves more refined regional division than the preset logic to accurately classify the seats corresponding to the radar processing point cloud. By improving the signal-to-noise ratio, unsupervised clustering self-labeling, and refined machine learning classification, it is possible to more accurately judge the presence and location of people in closed environments such as cabins. In addition, compared with rule-based regional judgment, the model construction and processing process reduces the design of rules, reduces the difficulty of parameter adjustment, and increases the convenience and applicability of the method.

[0262] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0263] The present application also provides an integrated circuit, which may include a radio frequency module, an analog signal processing module, and a digital signal processing module connected in sequence; the radio frequency module is used to generate a radio frequency transmit signal and receive an echo signal; the analog signal processing module is used to down-convert the echo signal to obtain an intermediate frequency signal; the digital processing module is used to perform analog-to-digital conversion on the intermediate frequency signal to obtain a digital signal; and the digital signal is processed based on the target detection method in the embodiment of the present application to achieve the purpose of target detection. For example, the integrated circuit may be a millimeter wave radar chip (chip or die). The digital processing module may include a baseband module and a main control module, etc., and each module may be configured to execute the corresponding steps in the target detection method in the above embodiment.

[0264] In some optional embodiments, the integrated circuit may be an AiP (Antenna-In-Package) chip structure, an AoP (Antenna-On-Package) chip structure, or an AoC (Antenna-On-Chip) chip structure.

[0265] According to other embodiments of the present application, an electromagnetic wave sensor is also proposed. The electromagnetic wave sensor may include an antenna and an integrated circuit as described above. The integrated circuit is electrically connected to the antenna for transmitting and receiving electromagnetic wave signals. For example, the electromagnetic wave sensor may include: a carrier, an integrated circuit and an antenna as described in any of the above embodiments, etc. The integrated circuit may be provided on the carrier; the antenna may be provided on the carrier, or integrated with the integrated circuit as an integral device provided on the carrier (i.e., the antenna may be an antenna provided in an AiP, AoP or AoC structure in this case); the integrated circuit is connected to the antenna (i.e., the sensor chip or integrated circuit does not have an integrated antenna in this case, such as a conventional SoC, etc.) for transmitting and receiving electromagnetic wave signals. The carrier may be a printed circuit board PCB, and the corresponding transmission line may be a PCB trace.

[0266] It should be noted that the electromagnetic waves in this application may include radio waves and light waves. Radio waves include short waves, medium waves, long waves and microwaves, etc. Microwaves include centimeter waves (i.e. electromagnetic waves in the range of 3GHz to 30GHz, such as electromagnetic waves in the range of 3.1GHz to 10.6GHz, electromagnetic waves in the 24GHz frequency band, etc.) and millimeter waves (i.e. electromagnetic waves in the range of 30GHz to 300GHz, such as electromagnetic waves in the 60GHz frequency band, electromagnetic waves in the 77GHz frequency band (such as 77GHz to 81GHz, etc.)). Light waves may include ultraviolet rays, visible light, infrared rays, and lasers, etc., among which the electromagnetic wave frequency band of lasers is (3.846 to 7.895)*10^5GHz, that is, lasers are included in part of the frequency bands of ultraviolet rays and visible light.

[0267] An embodiment of the present application provides a device, which may include: a device body; and an electromagnetic wave sensor as described above, which is arranged on the device body; wherein the electromagnetic wave sensor is used for target detection and / or communication to provide reference information for the operation of the device body.

[0268] An embodiment of the present application also provides an electronic device that can be expressed in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to, at least one processing unit, at least one storage unit, a bus connecting different system components (including a storage unit and a processing unit), a display unit, and the like. The storage unit stores program code, and the program code can be executed by the processing unit so that the processing unit executes the method described in this specification according to various exemplary embodiments of the present application. The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only memory unit (ROM).

[0269] The storage unit may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0270] The bus can represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0271] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter may communicate with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0272] For example, the electronic device in the embodiment of the present application may further include: a device body; and an electromagnetic wave sensor as described in any of the above embodiments, which is arranged on the device body; wherein the electromagnetic wave sensor can be used to realize functions such as target detection and / or wireless communication.

[0273] Specifically, based on the above embodiment, in an optional embodiment of the present application, the electromagnetic wave sensor can be disposed outside the device body or inside the device body, and in other optional embodiments of the present application, the electromagnetic wave sensor can be disposed partially inside the device body and partially outside the device body. This embodiment of the present application is not limited to this, and the specific configuration may vary depending on the circumstances.

[0274] In an optional embodiment, the above-mentioned device body can be a component and product used in fields such as smart cities, smart homes, transportation, smart homes, consumer electronics, security monitoring, industrial automation, cabin detection (such as smart cockpits), medical equipment and health care. For example, the device body can be intelligent transportation equipment (such as cars, bicycles, motorcycles, ships, subways, trains, etc.), security equipment (such as cameras), liquid level / flow rate detection equipment, smart wearable devices (such as bracelets, glasses, etc.), smart home devices (such as sweeping robots, door locks, televisions, air conditioners, smart lights, etc.), various communication devices (such as mobile phones, tablets, etc.), as well as gates, smart traffic lights, smart signs, traffic cameras and various industrial robotic arms (or robots), etc. It can also be various instruments for detecting life characteristic parameters and various devices equipped with the instruments, such as life characteristic detection in car cabins, indoor personnel monitoring, smart medical equipment, consumer electronic devices, etc.

[0275] An embodiment of the present application further provides a non-transitory computer-readable storage medium having computer-readable instructions stored thereon. When the instructions are executed by a processor, the processor executes the feeder unequal length compensation method as described above.

[0276] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software or by combining software with necessary hardware. The technical solution according to the embodiment of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiment of the present application.

[0277] The software product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0278] Computer-readable storage media may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0279] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0280] The computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the computer-readable medium implements the aforementioned functions.

[0281] Those skilled in the art will appreciate that the modules described above can be distributed in the device according to the description of the embodiment, or can be modified accordingly to be used in one or more devices that are different from the embodiment. The modules of the above embodiment can be combined into one module or further divided into multiple submodules.

[0282] According to an embodiment of the present application, a computer program is proposed, including a computer program or an instruction, which, when executed by a processor, can execute the method described above. In an optional embodiment, the above-mentioned integrated circuit can be a millimeter-wave radar chip. The type of digital function module in the integrated circuit can be determined according to actual needs. For example, in a millimeter-wave radar chip, the data processing module can be used for functions such as range-dimensional Doppler transform, velocity-dimensional Doppler transform, constant false alarm detection, direction of arrival detection, point cloud processing, etc., to obtain information such as the distance, angle, speed, height, micro-Doppler motion characteristics, shape, size, surface roughness and dielectric properties of the target.

[0283] It should be noted that radio devices can achieve functions such as target detection and / or communication by transmitting and receiving radio signals to provide detection target information and / or communication information to the device body, thereby assisting or even controlling the operation of the device body.

[0284] For example, when the above-mentioned device body is applied to an advanced driver assistance system (i.e., ADAS), the radio device (such as millimeter-wave radar) as an on-board sensor can assist the ADAS system to realize application scenarios such as adaptive cruise control, automatic braking assistance (i.e., AEB), blind spot detection warning (i.e., BSD), assisted lane change warning (i.e., LCA), reversing assistance warning (i.e., RCTA), parking assistance, rear vehicle warning, collision avoidance, pedestrian detection, and in-cabin liveness detection (i.e., CPD).

[0285]

[0286] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0287] The above-described embodiments merely express preferred embodiments of the present invention and the technical principles employed. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It is apparent to those skilled in the art that various changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, the present invention may also include more other equivalent embodiments. The scope of protection of the present invention patent is determined by the scope of the appended claims.

Claims

1. A target detection method, characterized in that: include: Performing FFT processing on the echo signal to generate a feature vector of the echo signal; The feature vector is processed based on a machine learning model to achieve target detection in the region of interest.

2. The target detection method according to claim 1, characterized in that: The feature vector includes energy characteristics representing the echo signal at different distance units, or the feature vector includes energy characteristics representing the echo signal at different Doppler dimensions at different distance units.

3. The target detection method according to claim 2, characterized in that: The feature vector includes energy characteristics of the echo signal in different Doppler dimensions of different distance units, and the performing FFT processing on the echo signal may include: Perform FFT processing on the echo signal to generate a range-Doppler spectrum; A beam synthesis process is performed according to the range-Doppler spectrum to obtain a feature vector of the echo signal.

4. The target detection method according to claim 3, characterized in that: The method of performing FFT processing on the echo signal to generate a range-Doppler spectrum includes: Performing distance-dimensional FFT processing on the echo signal; Performing sliding windowing on the result obtained by the range dimension FFT processing, and performing 2D FFT on the windowed data to obtain the range-Doppler spectrum; The length of the sliding window corresponds to a duration of the same order of magnitude as the period of the periodic motion of the target.

5. The target detection method according to claim 3, characterized in that: The beam synthesis processing according to the range-Doppler spectrum is implemented by the following expression: Wherein, P(r,v,b) is the energy characteristic of the vth Doppler unit of the rth range unit based on beam b, sv(c,b) is the steering vector of beam b on virtual channel c, x(c,r,v) is the data corresponding to the vth Doppler unit of the rth range unit on virtual channel c in the range-Doppler spectrum, N c is the total number of beams.

6. The target detection method according to claim 2, characterized in that: The feature vector includes energy features characterizing the echo signal in different Doppler dimensions of different distance units, and the performing FFT processing and beamforming processing on the echo signal includes: Perform range FFT processing on the echo signal; Perform beam synthesis processing on the result obtained by range FFT processing; Sliding windowing is performed on the result of the beamforming process, and 2D FFT is performed on the windowed data.

7. The target detection method according to claim 2, characterized in that: The feature vector includes energy characteristics that characterize the echo signal at different distance units, and the performing FFT processing and beamforming processing on the echo signal includes: Perform range FFT processing on the echo signal; The beamforming process is performed on the result obtained by the range FFT process.

8. The target detection method according to any one of claims 3 to 7, characterized in that: The beamforming comprises: Determining a target direction pointing to the region of interest and generating a steering vector according to the target direction; Beam synthesis is performed according to the steering vector.

9. The target detection method according to claim 8, characterized in that: The determining a target direction pointing to the region of interest includes: The target direction is determined according to the azimuth and elevation angle of the region of interest relative to the radar.

10. The target detection method according to claim 8, characterized in that: The determining a target direction pointing to the region of interest includes: Acquire measured data of the target in the region of interest; Clustering is performed according to the target measured data, so as to use the azimuth and elevation angles of the cluster center points in each of the regions of interest as the target direction.

11. The target detection method according to claim 8, characterized in that: The generating of the steering vector according to the target direction is realized by the following expression: sv(c,b)=exp(1j·2π / λ·(x c cosφ b sinθ b +z c sinφ b )), Where sv(c,b) is the steering vector of beam b on virtual channel c, λ is the central wavelength of the radar detection signal, and x c and z c are the horizontal and vertical positions of the antenna corresponding to the virtual channel c, θ b is the azimuth corresponding to the target direction, φ b is the pitch angle corresponding to the target direction.

12. The target detection method according to any one of claims 2 to 7, characterized in that: The step of generating a feature vector of the echo signal comprises: According to a preset rule, the energy features obtained by performing FFT processing and beamforming processing on the echo signal are rearranged to obtain the feature vector.

13. The target detection method according to claim 12, characterized in that: The preset rules include: The data with the same first dimension are arranged continuously in the changing direction of the second dimension, and the whole data are arranged continuously in the changing direction of the first dimension; or, The data with the same third dimension and the same fourth dimension are arranged continuously in the changing direction of the fifth dimension, the data with the same third dimension are arranged continuously in the changing direction of the fourth dimension, and the whole is arranged continuously in the changing direction of the third dimension; The first dimension and the second dimension are different dimensions in the beam dimension and the distance dimension, respectively; the third dimension, the fourth dimension and the fifth dimension are different dimensions in the beam dimension, the distance dimension and the Doppler dimension, respectively.

14. The target detection method according to any one of claims 2 to 7, characterized in that: The step of generating a feature vector of the echo signal comprises: The energy characteristics obtained by performing FFT processing and beamforming processing on the echo signal are logarithmically normalized.

15. The target detection method according to claim 14, characterized in that: The logarithmic normalization of energy features obtained by performing FFT processing and beamforming processing on the echo signal includes: The energy characteristics obtained by performing FFT processing and beamforming processing on the echo signal are logarithmized, and the logarithm result is linearly normalized using preset upper and lower bound constraints.

16. The target detection method according to claim 15, characterized in that: The energy characteristics obtained by performing FFT processing and beamforming processing on the echo signal are logarithmized, and the result after logarithmization is linearly normalized using preset upper and lower bound constraints, which is achieved by the following expression: Among them, P norm (r, v, b) is the logarithmic normalization result, P(r, v, b) is the energy characteristic of beam b at range unit r and Doppler unit v, a0, b0, P max and P min These are preset parameters.

17. The target detection method according to any one of claims 2 to 7, characterized in that: The machine learning model is a deep learning model. Before processing the feature vector based on the machine model, the method further includes: Obtaining a training feature vector, wherein the training feature vector carries a one-hot encoding label and is used to characterize energy characteristics of a signal at different distance units; The training feature vector is processed by a preset data enhancement method to obtain a training set, wherein the preset data enhancement method includes at least one of the following: randomly adding white noise, flipping the data along the Doppler dimension, and translating the data along the distance dimension; The deep learning model is trained according to the training set.

18. The target detection method according to any one of claims 2 to 7, characterized in that: The target detection result is a confidence matrix, and the elements in the confidence matrix correspond to at least one of the following information: the confidence of detecting the target, the confidence that the target exists in the corresponding region of interest, the confidence that an adult target exists in the corresponding region of interest, and the confidence that a child target exists in the corresponding region of interest.

19. The target detection method according to any one of claims 2 to 7, characterized in that: The feature vector includes energy features characterizing the echo signal at different distance units within a preset distance range and / or a preset Doppler range.

20. The target detection method according to claim 1, characterized in that: The machine learning model includes a neural network model, and the processing of the feature vector based on the machine model includes: Normalizing a preset number of frame data, and inputting the normalized results into a pre-trained neural network model; or A preset number of frame data are normalized, the normalized result is converted into a complex number, and the complex number or the real part of the complex number or the imaginary part of the complex number is input into the neural network model.

21. The method according to claim 20, characterized in that The normalizing process of the preset number of frame data includes: performing normalization process using the following formula: Among them, P norm (r, v, b) is the normalized result of beam b range unit r chirp v, P(r, v, b) is the power of DBF data of beam b range unit r chirp v, b0 and a0 are preset normalization parameters, P max and P min are the preset upper and lower limits of P(r,v,b) respectively.

22. The method according to claim 21, characterized in that The b0 is the mean or median of all possible log2(P(r,v,b)) values, and a0 is the difference between the 95% quantile and the 5% quantile of all possible log2(P(r,v,b)) values ​​divided by 256.

23. The method according to claim 20, characterized in that The converting the normalized result into a complex number includes: converting using the following formula: real (r,v,b)=Re[P norm (r,v,b)·exp(1j·φ(r,v,b))]z img (r,v,b)=Im[P norm (r,v,b)·exp(1j·φ(r,v,b))] Among them, z real and z img are the real and imaginary parts after conversion, P norm (r,v,b) is the result after normalization, and φ(r,v,b) is the phase of chirp v of beam b range unit r.

24. The method according to claim 23, characterized in that The neural network model is a complex neural network model, and / or the neural network model includes a complex convolutional layer, a complex nonlinear activation layer, a pooling layer, a complex fully connected layer and an absolute value layer.

25. The target detection method according to claim 1, characterized in that: The performing FFT processing on the echo signal to generate a feature vector of the echo signal includes: After performing distance-dimensional FFT processing based on the echo signal, 1D-FFT data is obtained, and multi-frame joint processing is performed on the 1D-FFT data to obtain an RD spectrum as the feature vector.

26. The target detection method according to claim 25, characterized in that: The model-based processing of the feature vector to achieve target detection in the region of interest includes: Inputting the target point cloud data into a pre-trained first machine learning classifier to obtain a first detection result, wherein the first detection result indicates whether the candidate target detection point is a valid candidate target detection point, and the target point cloud data includes coordinate data; Among them, the first machine learning classifier is any one or more of the following classifiers: support vector machine, random forest, decision tree, mixed Gaussian model, KNN, hidden Markov, multi-layer perceptron.

27. The method according to claim 26, characterized in that The data input into the first machine learning classifier also includes one or more of the following: distance data, constant false alarm signal-to-noise ratio, amplitude value, Doppler frequency, azimuth median angle and pitch angle.

26. [Corrected 09.09.2024 in accordance with Rule 91] 28. [Corrected 09.09.2024 in accordance with Rule 91] The target detection method according to claim 25, characterized in that The model-based processing of the feature vector to achieve target detection in the region of interest includes: Processing the target point cloud data, extracting region information of candidate target detection points, inputting the region information into a pre-trained second machine learning classifier to obtain a second detection result, wherein the second detection result indicates whether there is a target in a preset region; Among them, the second machine learning classifier is any one or more of the following classifiers: support vector machine, random forest, decision tree, mixed Gaussian model, KNN, hidden Markov, multi-layer perceptron.

29. [Corrected 09.09.2024 in accordance with Rule 91] An integrated circuit, characterized in that It includes a radio frequency module, an analog signal processing module and a digital signal processing module connected in sequence; Wherein, the RF module is used to generate RF transmission signals and receive RF reception signals; The analog signal processing module is used to down-convert the RF received signal to obtain an intermediate frequency signal; The digital signal processing module is used to perform analog-to-digital conversion on the intermediate frequency signal, and to process the digital data obtained by the analog-to-digital conversion using the target detection method described in any one of claims 1 to 28 to achieve target detection.

30. [Corrected 09.09.2024 in accordance with Rule 91] An electromagnetic wave sensor, characterized in that: include: Carrier; The integrated circuit of claim 29, disposed on a carrier; An antenna is arranged on the carrier, or the antenna and the integrated circuit are integrated into one device and arranged on the carrier; The integrated circuit is connected to the antenna and is used to transmit a radio frequency transmission signal and / or receive a radio frequency reception signal.

31. [Corrected 09.09.2024 in accordance with Rule 91] A terminal device, characterized in that, include: Equipment body; and an electromagnetic wave sensor as claimed in claim 30 disposed on the device body; The electromagnetic wave sensor is used for target detection and / or communication to provide reference information for the operation of the device body.

27. [Corrected 09.09.2024 in accordance with Rule 91] 28. [Corrected 09.09.2024 in accordance with Rule 91] 29. [Corrected 09.09.2024 in accordance with Rule 91]

Images