Target detection method, integrated circuit, sensor and equipment

Abstract

The embodiment of the invention relates to the technical field of electromagnetic wave sensors, and discloses a target detection method, an integrated circuit, a sensor and equipment. The target detection method comprises the following steps: carrying out distance dimension FFT (Fast Fourier Transform) on frame data; extracting target data from the frame data obtained after the distance dimension FFT processing; windowing the target data according to a sliding window, and performing digital signal processing on the windowed data; wherein the duration, corresponding to the length of the sliding window, of the same order of magnitude as the period of the periodic motion of the target. Through a multi-frame joint processing technology, a scheme of target detection in space areas such as an automobile cabin, a room and a factory building is realized, the detection rate can be effectively improved, the false alarm target number is reduced, and meanwhile, the angle estimation accuracy can be greatly improved. Therefore, special targets or weak targets such as infants and the like in the cabin can be accurately detected, and applications such as CPD and SBR are realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of electromagnetic wave sensors, and in particular to a target detection method, an integrated circuit, a sensor, and a device. Background Art

[0002] When performing target detection in a sealed or relatively sealed space area, such as detecting targets in a car cabin, there may be technical problems such as low detection rate, a large number of false alarm targets, and inaccurate angle estimation. Summary of the invention

[0003] The embodiments of the present application provide a target detection method, integrated circuit, sensor and device, which are at least conducive to realizing a solution for target detection in spatial areas such as automobile cabins, indoors, and factory buildings through multi-frame joint processing technology. While effectively improving the detection rate and reducing the number of false alarm targets, it can also greatly improve the accuracy of angle estimation and other effects, thereby enabling special targets or weak targets such as infants and young children in the cabin to be accurately detected, thereby realizing applications such as CPD (Child Presence Detection) and SBR (Safety Belt Reminder).

[0004] According to some embodiments of the present application, a first aspect of the embodiments of the present application provides a target detection method, comprising: performing distance-dimensional FFT on frame data; extracting target data from the frame data obtained after distance-dimensional FFT processing; sliding windowing the target data, and performing digital signal processing on the windowed data to obtain target information; wherein the length of the sliding window corresponds to a duration of the same order of magnitude as the period of the periodic motion of the target, that is, the above-mentioned target information may include information such as the period, distance, angle, etc. of the periodic motion of the target.

[0005] According to some embodiments of the present application, the second aspect of the embodiments of the present application further provides a target detection method, including: performing target detection according to the target detection method as described in the first aspect of the embodiments of the present application to obtain a target; based on the targets falling into each preset area, verifying the targets in each preset area, the preset areas being obtained by dividing the detection space; based on the targets that pass the verification, outputting the detection results for each preset area.

[0006] According to some embodiments of the present application, the third aspect of the embodiments of the present application also provides a target detection method, including: performing distance-dimensional FFT on frame data; extracting target data from the frame data obtained after distance-dimensional FFT processing; windowing the target data according to the sliding windows corresponding to the periods of different vital sign parameters of the living body, and performing digital signal processing on the windowed data corresponding to sliding windows of different lengths; wherein, the periods of the vital sign parameters corresponding to different sliding windows are different, and the length of each sliding window has a duration of the same order of magnitude as the period of the vital sign parameter.

[0007] According to some embodiments of the present application, the fourth aspect of the embodiments of the present application further provides an integrated circuit, comprising a radio frequency module, an analog signal processing module and a digital signal processing module connected in sequence; wherein the radio frequency module is used to generate a radio frequency transmission signal and receive a radio frequency reception signal; the analog signal processing module is used to down-convert the radio frequency reception signal to obtain an intermediate frequency signal; the digital signal processing module is used to perform analog-to-digital conversion on the intermediate frequency signal, perform distance-dimensional FFT on the frame data obtained by the analog-to-digital conversion, extract target data from the frame data obtained after the distance-dimensional FFT processing, window the target data according to a sliding window, and perform digital signal processing on the windowed data; wherein the length of the sliding window corresponds to a duration of the same order of magnitude as the period of the target's periodic motion.

[0008] According to some embodiments of the present application, the fifth aspect of the embodiments of the present application further provides an electromagnetic wave sensor, comprising: a carrier; the integrated circuit as described in the fourth aspect of the embodiments of the present application, which is arranged on the carrier; an antenna, which is arranged on the carrier, or the antenna and the integrated circuit are integrated into an integrated device and arranged on the carrier; wherein the integrated circuit is connected to the antenna for transmitting radio frequency transmission signals and / or receiving radio frequency reception signals.

[0009] According to some embodiments of the present application, the sixth aspect of the embodiments of the present application further provides a device, comprising: a device body; and an electromagnetic wave sensor as described in the fifth aspect of the embodiments of the present application, which is arranged on the device body; wherein the electromagnetic wave sensor is used for target detection and / or communication to provide reference information for the operation of the device body.

[0010] The technical solution provided in the embodiment of the present application has at least the following advantages:

[0011] By using multiple frames of data with similar physiological cycles, it is possible to better detect targets in combination with the characteristics of the human physiological cycle, avoid errors and omissions, and eliminate the influence of multipath, which is beneficial for processing and improving accuracy. By extracting target data instead of using all data, processing efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0013] Figure 1 This is a flow chart of realizing target detection based on SISO-combine in an embodiment of the present application;

[0014] Figure 2 A flow chart of a target detection method provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of the working cycle of the radar provided in an embodiment of the present application;

[0016] Figure 4 A waveform diagram of a detection signal of an FMCW radar provided in an embodiment of the present application;

[0017] Figure 5 Another flow chart of the target detection method provided in the embodiment of the present application;

[0018] Figure 6 Another flow chart of the target detection method provided in the embodiment of the present application;

[0019] Figure 7 Another flow chart of the target detection method provided in the embodiment of the present application;

[0020] Figure 8 Another flow chart of the target detection method provided in the embodiment of the present application;

[0021] Fig. 9 A schematic diagram of data reading involved in the target detection method provided in an embodiment of the present application;

[0022] Fig.10 Another flow chart of the target detection method provided in the embodiment of the present application;

[0023] Fig.11 Another flow chart of the target detection method provided in the embodiment of the present application;

[0024] Fig.12 A schematic diagram of data movement involved in the target detection method provided in an embodiment of the present application;

[0025] Fig.13 Another flow chart of the target detection method provided in the embodiment of the present application;

[0026] Fig.14 A schematic diagram of the division of a preset area involved in the target detection method provided in an embodiment of the present application;

[0027] Fig.15 Another flow chart of the target detection method provided in the embodiment of the present application;

[0028] Fig.16 Another flow chart of the target detection method provided in the embodiment of the present application;

[0029] Fig.17 A schematic diagram of the distribution of target points in a preset area involved in the target detection method provided in an embodiment of the present application;

[0030] Fig.18 Another flow chart of the target detection method provided in the embodiment of the present application;

[0031] Fig.19 This is a flow chart of implementing target detection based on per-channel in an embodiment of the present application;

[0032] Fig. 20 It is a flowchart of realizing target detection based on frame-level FFT combined with DAE CFAR in an embodiment of the present application;

[0033] Fig.21 Based on Fig. 20 A schematic diagram of azimuth or elevation CFAR in the target detection process shown;

[0034] Fig. 22 This is a schematic diagram of a process for post-processing a target in an embodiment of the present application;

[0035] Fig.23 This is a flow chart of post-processing a target in combination with a DL algorithm in an embodiment of the present application;

[0036] Fig.24 This is a flow chart of a target detection method combined with a DL algorithm in an embodiment of the present application;

[0037] Fig.25 It is a flowchart of another target detection method combined with DL algorithm in an embodiment of the present application;

[0038] Fig.26 Schematic diagram of the flow of another target detection method combined with DL algorithm in an embodiment of the present application. DETAILED DESCRIPTION

[0039] When detecting targets in sealed or relatively sealed spatial areas, the scheme of target detection in sealed spatial areas such as cabins, indoors, and factories is implemented through inter-frame accumulation, which can effectively improve the detection rate, reduce the number of false alarm targets, and greatly improve the accuracy of angle estimation, so that special targets or weak targets such as infants in the cabin can be accurately detected, and applications such as child presence detection (CPD) and safety belt reminder (SBR) can be realized. In the specific implementation process, it can also be combined with other technologies such as deep learning detection positioning and recognition technology based on multi-frame joint processing technology to achieve accurate detection of targets in the cabin.

[0040] In the multi-frame joint processing scheme, after obtaining the range-Doppler map by sliding window fast Fourier transform (Fast Fourier Transform, FFT) of multi-frame data, it can also be processed by finite impulse response (Finite Impulse Response, FIR) or other complex time-frequency transformation, and then the area of ​​interest is subjected to constant false alarm detection (Constant False-Alarm Rate, CFAR), direction of arrival estimation (Direction Of Arrival, DOA) and other processing operations, and can be combined with the set area judgment logic and area parameters, etc., to achieve the detection and positioning of living targets such as adults, children, pets, or other non-living targets. CFAR can be NR-CFAR in the Doppler dimension, or RD-CFAR, or DAE (Doppler-Azimuth-Elevation)-CFAR, etc. At the same time, clustering, false alarm suppression, multiple processing point cloud association and other post-processing can be used after CFAR and DOA. The regional parameters can be determined by adopting at least one of the following schemes, or a combination of at least two schemes, including clustering operation on the point cloud detected after a certain sliding window multi-frame processing, outlier detection and elimination operation on the point cloud detected after a certain sliding window multi-frame processing, etc.

[0041] Among them, when applied to a closed space area or a relatively closed space area, the space area can be pre-divided to achieve the definition and detection of different areas of interest (divided areas). For example, when detecting targets in the cabin, the radar monitoring area can be divided into seat intervals and aisle intervals, etc., and then corresponding parameter types and thresholds are preset for different types of areas, and the corresponding processing steps can be combined to achieve accurate detection of targets of interest in specific areas. For example, for a family car, the space area in the car can generally be simply divided into head space, rear space and luggage space. If the rear space is the key monitoring area, the rear space can be further divided into seat intervals and aisle intervals, etc. At the same time, the seat interval can also be divided into a corresponding number of seat interval units based on the seat. Similarly, the aisle interval can also be divided into a corresponding number of aisle interval units corresponding to the above-mentioned seat interval units. For example, for a five-seat family car, for the three seats in the rear space, three seat interval units and corresponding three aisle interval units can be divided. At the same time, corresponding parameter types and thresholds can be preset for different types of regions (or intervals, or interval units), and adaptive signal data processing methods and steps can be used to achieve accurate detection of the target of interest (specific target) in a specific interval or interval unit. Among them, adjacent interval units may have a partial overlapping area, or may be adjacent or separated by a gap of a preset width.

[0042] When combined with the detection, positioning and identification scheme of deep learning, its implementation can be: after performing 1D-FFT (such as range-dimensional FFT) processing on the echo signal, continue with multi-frame sliding window FFT, and perform N-point two-dimensional digital beam forming (2D Digital Beam Forming, 2D-DBF) processing to form an N-channel RD (Range-Doppler) spectrum, and then use the deep learning model to perform monitoring, positioning and identification operations based on the N-channel RD spectrum; or, after performing 1D-FFT processing on the echo signal and continuing N-point 2D-DBF processing, use the deep learning model to perform monitoring, positioning and identification operations based on the N-channel RD spectrum; or, first perform 1D-FFT processing on the echo signal and then continue N-point 2D-DBF processing, continue multi-frame sliding window FFT to form an N-channel RD spectrum, and then use the deep learning model to perform monitoring, positioning and identification operations based on the N-channel RD spectrum. Among them, the above-mentioned N can be the number of interval units; for example, for a rear row interval with three seats in the rear row, it is divided into three seat interval units and corresponding three aisle interval units, then N can be equal to 6.

[0043] When combined with fast and slow time processing, its implementation can be: based on single-frame fast and slow time processing and CFAR detection, angle measurement point cloud and other regional judgment to achieve target detection, or based on single-frame fast time processing and using Capon and other algorithms to perform DOA (direction of arrival estimation) and other operations, so as to form a range-azimuth (RA) diagram and then extract the energy and other features in the predetermined area for identification and judgment to achieve target detection, or based on the phase of the docked return echo after filtering and then detecting the human body's breathing and heartbeat to achieve target detection. For example, when the target detection is achieved by regional judgment based on single-frame fast and slow time processing and CFAR detection, angle measurement point cloud, after the echo signal is processed by 1D-FFT (such as range dimension FFT), time correlation analysis can be performed based on the complex echo data to achieve human target discrimination, and then the position can be judged by combining the point cloud information, so as to finally achieve the purpose of cabin target detection. For example, when target detection is achieved by performing regional judgment based on the point cloud obtained by single-frame fast and slow time processing and CFAR detection and angle measurement, after performing 1D-FFT (such as range-dimensional FFT) processing on the echo signal, time correlation analysis can be performed based on the complex echo data to achieve human target discrimination. Subsequently, position judgment can be performed by combining point cloud information, so as to ultimately achieve the purpose of in-cabin target detection.

[0044] Based on the above scheme, after testing with real data, the top-mounted radar installation scenario can effectively achieve a high detection rate, as well as extremely low missed detection rate and false alarm rate. For example, when detecting targets in the cabin, it can achieve a detection rate of more than 99%, a missed detection rate of less than 0.5%, and a false alarm rate of less than 1%.

[0045] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present application, many technical details are provided in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented.

[0046] The following division of each embodiment is for the convenience of description and should not constitute any limitation on the specific implementation of the present application. Each embodiment can be combined with each other and referenced to each other under the premise of no contradiction. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0047] The embodiment of the present application provides a target detection method. It performs digital signal processing on multiple frames of data with similar periods of the periodic motion of the target, so that the target detection can be better performed in combination with the characteristics of the target motion period, avoiding errors and omissions while excluding the influence of multipath, etc., which is conducive to processing and improving accuracy, and by extracting target data instead of using all data, the processing efficiency is improved. The following will be described in conjunction with different embodiments.

[0048] In order to facilitate those skilled in the art to better understand the target detection method provided by this embodiment, the complete digital signal processing process implemented by the method in a SISO-combine radar will be used as an example for explanation below.

[0049] After receiving the echo signal through the antenna and performing analog-to-digital conversion and sampling, the radar will obtain a digital signal, which will then be processed as follows: Figure 1 (Take the target detection of frequency modulated continuous wave (FMCW) radar as an example) The main control module (for example, it can be implemented based on a microcontroller unit (MCU)) and the baseband (BB) module (for example, it can be implemented based on a baseband chip) shown in the figure are processed to realize target detection. Specifically, in Figure 1In the process, the digital signal will be subjected to range DC removal and range Fourier transform (Range FFT, which can also be expressed as 1D-FFT or fast time FFT) in the baseband module. Afterwards, the baseband module can further process the data, for example, combine multiple chirp data of one frame by complex averaging, etc. Of course, no further processing is required. After that, the signal processed by the baseband module will be sent to the main control module and accumulated in multiple frames. When the data accumulates to a preset number of frames, such as 128 frames, the main control module will send the accumulated data of multiple frames from the CPU static random access memory (SRAM) to the baseband module, and continue to perform Doppler DC removal, frame Fourier transform (Frame FFT), channel accumulation (non-coherent integration processing, channel accumulation, etc.) based on the baseband module, and then perform constant false alarm detection (such as peak-based CFAR (detection with peak selection)) based on the noise obtained by noise estimation (such as using a variance-based noise estimator) in the baseband module, and finally perform angle detection and target classification and other processing to obtain the target's distance, speed and / or angle information. When performing angle detection, azimuth (Azimut) estimation and elevation (Elevation) estimation can be performed based on the CFAR result, which can be achieved through technologies such as digital beamforming (DBF) and direction of arrival estimation.

[0050] The following will Figure 1 The specific implementation of the digital signal processing shown is described.

[0051] The distance-dimensional DC removal is aimed at the data corresponding to each chirp in each frame of data. At this time, the distance-dimensional DC removal is specifically as follows: the data collected by each receiving (RX) channel of each chirp is averaged along the fast time dimension, and then the DC component of all sampling points of each RX channel is subtracted.

[0052] The distance dimension FFT can be realized by windowing the data after the distance dimension DC is removed and then performing 1D-FFT.

[0053] It should be emphasized that in the process of target detection, each chirp data in each frame of data obtained by radar through ADC or sampling based on the echo signal is sent to the main control module through the above operation. The data is sent to the main control module by moving the data from Direct Memory Access (DMA) to CPU SRAM.

[0054] Doppler dimension DC removal can be achieved by calculating the complex average along the frame dimension of multiple frame data accumulated on the main control module (i.e. averaging the data of different frames in the same distance unit (bin) of the same channel), obtaining the average values ​​of different distances in different channels as the DC components of the corresponding distance units of the corresponding channels, and subtracting the respective DC components from each distance unit of each channel.

[0055] The inter-frame Fourier transform can be implemented by windowing the data after the Doppler dimension DC is removed along the frame dimension and performing 2D-FFT along the frame dimension, at which time the range dimension-Doppler information of each transmitting and receiving channel will be obtained.

[0056] Inter-channel accumulation, taking incoherent accumulation as an example, can be implemented by the following expression:

[0057]

[0058] or,

[0059]

[0060] Wherein, P(r,v) and A(r,v) are the power and amplitude at the r-th range unit and the v-th Doppler unit, respectively, and s(r,t,a,v) is the complex value obtained by inter-frame Fourier transform of the signal echo of the r-th range unit and the v-th Doppler unit on the t-th transmission channel and the a-th receiving channel.

[0061] The constant false alarm detection based on the noise floor obtained by noise estimation has been described in the existing CFAR technology, and will not be described in detail here.

[0062] Angle detection can be achieved by performing azimuth digital beam forming (DBF) and DOA on the target point obtained by CFAR, as well as elevation DBF and DOA. DBF and DOA have been described in the existing DBF and DOA technologies, and will not be described here one by one.

[0063] Target classification can be achieved by counting the target points detected in each frame (or a preset amount of data, i.e. each time) according to a preset (or divided) area unit design, and the counting results can be judged according to preset rules to obtain whether there is a physical target of interest (such as an adult, child, infant or pet, etc.) in this processing and determine its location area.

[0064] It should be noted that for radars with baseband modules and main control modules, Figure 1 The above is only an example, that is, frame data accumulation, target classification (which can be implemented by Region HIST, classification, etc.) and other operations are performed in the main control module, and other operations are performed in the baseband module, such as range dimension DC removal and Doppler dimension DC removal, etc. In other embodiments, frame data accumulation can also be performed in the baseband module, and Doppler dimension DC removal can be performed in the main control module.

[0065] It should also be noted that Figure 1 Only caching 128 frames of data is used as an example for explanation. In other embodiments, 56 frames of data or 32 frames can also be cached. The embodiments of the present application are not limited to this and can be determined according to demand or hardware support capabilities. In this way, the cached multi-frame data will continue to perform Doppler dimension DC removal and inter-frame FFT to obtain a range-Doppler (Range-Doppler, RD) diagram.

[0066] It should also be noted that Figure 1 In the process shown, some steps can be skipped, such as distance-dimensional DC removal or distance-dimensional DC removal or windowing; some steps can be simplified, such as non-coherent accumulation can be skipped, and only one of the channels is subjected to CFAR detection, or only some channels are selected for non-coherent accumulation, and only these channels are subjected to CFAR detection.

[0067] To facilitate better understanding of those skilled in the art Figure 1 The process shown in the figure will be explained below through the processes of different target detection methods.

[0068] In some embodiments, the process of the target detection method is as follows Figure 2 As shown, the following steps are included:

[0069] Step 201: Perform distance dimension FFT on frame data.

[0070] Step 202: extract target data from the frame data obtained after the range-dimensional FFT processing.

[0071] Step 203, windowing the target data according to the sliding window, and performing digital signal processing on the windowed data.

[0072] The length of the sliding window corresponds to a duration of the same order of magnitude as the period of the periodic motion of the target.

[0073] To facilitate better understanding of those skilled in the art Figure 2 The steps of the target detection method described in the illustrated embodiment will be explained below.

[0074] In step 201, the frame data refers to the data corresponding to the echo signal formed after a frame of detection signal is reflected by the target. It can be understood that the target detection behavior performed by the radar is usually as follows: Figure 3 As shown in FIG. 1 , the radar first sends a detection signal in a detection cycle. Taking the detection of frequency modulated continuous wave (FMCW) radar as an example, Figure 4 As shown, a frame of detection signal will consist of multiple chirp signals ( Figure 4 The chirp signals shown have no idle periods for example only. In some cases, there may be a certain idle period between chirp signals. When the detection signal starts to be transmitted, the radar is ready to receive the echo signal reflected by the target. The echo signal corresponding to a frame of detection signal will be processed by the aforementioned analog-to-digital conversion, distance dimension DC removal, etc. to obtain the above frame data. Further, each frame data is subjected to distance dimension FFT as described above to implement step 201, which will not be described one by one here.

[0075] In step 202, the target data is not limited in the embodiment of the present application, and it can be any data content belonging to the corresponding frame data. For example, when the storage resources are sufficient and the computing power is sufficient, the target data can be extracted by extracting all the data; or when the real-time performance of the target detection is expected to be the highest, at least part of the data or the data preprocessed on at least part of the data (such as the data obtained after preprocessing operations such as complex averaging, complex weighted averaging or complex summing, etc.) can be extracted, etc. The target data can be extracted according to the requirements and hardware conditions. For ease of understanding, the following will illustrate different implementations of extracting part of the data as the target data.

[0076] In some embodiments, Figure 5 As shown, the target data is extracted from the frame data obtained after the distance dimension FFT processing, which is achieved through the following steps:

[0077] Step 2021, extract the data corresponding to any one or specified index chirp period (such as the second chirp of each frame) from each frame data obtained after distance dimension FFT processing as the target data; wherein, when it is confirmed that the data corresponding to the predetermined rule chirp period is used as the target data, the rule of the chirp selected in each frame is the same.

[0078] In other words, using the data corresponding to a chirp cycle as the representation of the frame data makes the processing of the target data concise and easy to implement, and the selected data is more real, which can perform target detection more efficiently and is conducive to improving the real-time performance of target detection.

[0079] In some embodiments, Figure 6 As shown, extracting target data from the frame data obtained after distance dimension FFT processing can also be achieved through the following steps:

[0080] Step 2022, based on each frame of data obtained after the distance dimension FFT processing, determine the parameters corresponding to each distance unit in each frame of data to generate target data.

[0081] That is to say, multiple data of each distance unit in the frame data are compressed into corresponding parameters so that the overall characteristics of the frame data can be retained, so that the information that can be referred to when performing target detection based on the target data later is more comprehensive and the result obtained is more accurate.

[0082] Of course, the above embodiments are only provided to reduce the resources occupied by target data. In some embodiments, the validity of the data can also be considered to extract the target data. Specifically, the radar signal can usually cover a large range, but not all the ranges covered by the radar are areas that need attention. Processing the data corresponding to these areas will cause a waste of resources. Therefore, in some embodiments, such as Figure 7 As shown, extracting target data from the frame data obtained after distance dimension FFT processing can also be achieved through the following steps:

[0083] Step 2023, extracting data within a preset distance range from the frame data obtained after the distance dimension FFT processing as target data.

[0084] That is to say, in the frame data, the range of interest (represented by a preset distance range) is used as the target data. In this way, accurate target detection information can be retained and the storage resources and computing resources occupied by the target data can be reduced, thereby balancing the accuracy and real-time performance of target detection.

[0085] It should be noted that the retention of data in the range of interest is not limited to Figure 7 In some embodiments, it can also be implemented as shown. Figure 8 As shown, the target data is windowed according to the sliding window, and digital signal processing is performed on the windowed data, which is achieved through the following steps:

[0086] Step 2031, windowing the target data according to the sliding window, and performing digital signal processing on the data within the preset Doppler range in the windowed data.

[0087] It can also achieve Figure 7 The effect that can be achieved in the embodiment shown is that the dimensions of the two data are different, one is the distance dimension and the other is the Doppler dimension, so the selection method of the range of interest is different. Of course, it can also be as follows Figure 7 The embodiment shown extracts target data in the distance dimension. Figure 8 The illustrated embodiment extracts data in the Doppler dimension for digital signal processing, etc., which will not be described in detail here.

[0088] Of course, the above is only an example of the implementation method of step 202. In some embodiments, it can also be implemented by other methods. For example, the distance unit with a larger modulus between the distance units (similar to max-pooling) can be used as the target data, etc., which will not be described in detail here.

[0089] In step 203, the windowing method is not limited. For example, the window may be a sliding window, or a sliding window may be a fixed-length window, or a variable-length window, etc., which will not be listed one by one here. For ease of understanding, a fixed-length sliding window may be used as an example for explanation. However, it does not mean that only a fixed-length sliding window can be used. For example, a variable-length window may be changed according to the periodic movement of the target, such as changes in breathing frequency.

[0090] In some embodiments, Fig. 9 As shown, for the 1st to 129th frame data generated after the radar is started, taking the window length of the sliding window as 128 frames as an example, the data obtained by the sliding window for the first time is the 1st to 128th frame data ( Fig. 9 The data obtained the second time are the 2nd to 129th frame data ( Fig. 9 The data in the shaded part of the second row in ), that is, the sliding step of the sliding window is 1 frame.

[0091] Of course, the above are only examples. In other embodiments, the window length and sliding step size of the sliding window may also adopt other parameters, which will not be listed here one by one.

[0092] Furthermore, in step 203, the data signal processing is not limited. It is understandable that different data need to be processed for different requirements. For example, in some embodiments, the target detection effect may be achieved when the target point is detected. In some embodiments, it may also be necessary to Figure 1 As shown, target classification is performed to obtain further more accurate, reliable, and reference targets, etc., which will not be listed here one by one. Figure 1 The target detection process shown is used as an example, but it does not mean that it must be Figure 1 As shown in the figure, after being processed by CFAR and the like, target classification is still required.

[0093] like Figure 1 As shown, the digital signal processing process of target detection generally includes constant false alarm detection, a typical constant false alarm detection, that is, constant false alarm detection based on noise floor. It can be understood that at this time, the more accurate the estimation of the noise floor is, the better the constant false alarm detection effect is, and the more accurate the target detection result is. Therefore, in some embodiments, as Fig.10 As shown, windowing the target data according to the sliding window and performing digital signal processing on the windowed data can be achieved through the following steps:

[0094] Step 2032, windowing the target data according to the sliding window, and performing noise floor estimation on the windowed data within a preset range, wherein the upper limit of the preset range is determined according to the noise floor estimation result of the farthest distance unit.

[0095] Step 2034: Perform constant false alarm detection based on the estimated noise floor.

[0096] That is to say, by setting a preset range to constrain the noise floor estimation, an upper bound is constructed according to the noise floor estimation result of the farthest distance unit, which is beneficial to improving the accuracy of noise estimation.

[0097] In some examples, the noise floor is estimated within a preset range using the following expression:

[0098] n i '=min(n i ,α*n g );

[0099] Among them, n i ' is the noise floor estimation result of the ith distance unit within the preset range, n i is the initial noise floor estimation result of the ith distance unit, α is the preset parameter, α≥1, n g is the initial noise floor estimation result of the farthest distance unit.

[0100] It should be noted that the preset parameter α can be set and updated based on demand or engineering data, experience, etc.

[0101] It should also be noted that the above is only an exemplary description of the noise floor estimation. In some embodiments, the initial noise floor estimation result of the i-th distance unit can be directly selected as the noise floor result used in CFAR. The minimum value, maximum value, quantile, average, median, etc. of the initial noise floor estimation results of the last few regions of interest can also be selected as the noise floor estimation. Or, when estimating the noise floor, only some Doppler units of a certain distance unit are selected instead of all Doppler units, etc. The noise floor estimation methods are not listed one by one here.

[0102] also, Figure 1 The radar shown is a SISO-combine system, and when the radar includes multiple channels, the RD spectra of the multiple channels can also be integrated between channels, such as non-coherent integration and correlation integration, and the NVE module can be used to estimate the noise floor of each range bin in the accumulated data, wherein the noise floor estimation method can adopt any of the aforementioned noise floor estimation schemes, which will not be described one by one here.

[0103] In step 203, the data is windowed in a certain order. If the order is disrupted, the temporal association of the data is destroyed and it can no longer reflect the movement accumulated over a certain period of time, which is not conducive to subsequent target detection. Figure 1 As shown, data is accumulated in the main control module. When a certain amount of data is accumulated, the baseband module reads the data and continues to process it. In this process, the data writing order and reading order may be different due to the storage method. Therefore, in some embodiments, the data needs to be rearranged to ensure its correct order. For example, in some embodiments, assuming that the data is cached using Ring FIFO, at this time, Fig.11 As shown, the target detection method also includes the following steps:

[0104] Step 204, reading data from the Ring FIFO and reordering the read data.

[0105] Thus, the efficiency of subsequent processing can be improved by rearranging data.

[0106] In order to facilitate those skilled in the art to better understand the above data storage and rearrangement, the following will be combined with Fig.12 Provide explanation.

[0107] like Fig.12 As shown, assuming that the baseband module (shown as BB in the figure) performs distance dimension FFT on the frame data and obtains the target data in the range of interest (such as from the 5th distance unit to the 20th distance unit), the main control module (shown as MCU in the figure) will need to accumulate the following Fig.12 The two-dimensional matrix shown in FIG. 1 , the rows of the matrix correspond to the frame numbers, and the columns of the matrix correspond to the distance units.

[0108] Assuming that Ring FIFO is used in the main control module to store the above two-dimensional matrix, the storage process can be as follows:

[0109] Initialize a CPU SRAM with N f Columns, each column can store N r ×N rx ×N tx Number, where N f To design the maximum number of frames for inter-frame FFT, N r 、N rx and N tx The number of range units, the number of transmit (Tx) channels, and the number of receive (Rx) channels of interest are respectively. The buffer can be initialized to all 0s during initialization, and the FIFO head pointer p_head points to the beginning of the 0th column of the Ring-FIFO.

[0110] In the 0th frame, the corresponding data (of each distance unit, each Tx channel and Rx channel) is cached in the 0th column of the Ring-FIFO, and then p_head points to the 1st column.

[0111] Similarly, for the k-th frame data (k=1, 2, ..., 126), the corresponding data is cached in the k-th column of the Ring-FIFO, and then p_head points to the k+1-th column.

[0112] At the 127th frame, the corresponding data is cached in the 127th column of the Ring-FIFO; since 128 frames of data are already stored at this time, the data in the Ring-FIFO needs to be moved to the baseband module to perform the aforementioned inter-frame FFT processing. At this time, the data from the column pointed to by the p_head to the 127th column is first moved, and then the data from the 0th column to the column pointed to by the p_head is moved. Obviously, the order of reading data at this time is different from the order of writing, and it needs to be restored, that is, the data needs to be rearranged so that it is continuously distributed according to the frame dimension, thereby improving the computing efficiency on the baseband module.

[0113] After the move is completed, the pointer p_head of the Ring-FIFO points to column 0 again. Similarly, during the new data storage and move process, the data buffer of the 128th frame is stored in column 0 of the Ring-FIFO. During the move, the data of frames 1 to 127 will be moved to the baseband module first, and then the data of column 0 will be moved to the baseband module. The same goes for subsequent frames, which will not be described here.

[0114] It should be noted that in the above example, inter-frame processing is performed after all valid data is stored in the Ring-FIFO. In other embodiments, after storing part of the frame data, such as the 0th to 63rd frame data, the data in the Ring-FIFO can be moved to the baseband module in a similar manner to perform inter-frame FFT, etc., wherein, since the 64th to 127th columns are all initial values ​​assigned during initialization, it can be equivalent to the inter-frame FFT of the baseband module that fills the data sequence with zeros. Of course, the above is only an example, and does not mean that 128 frame data must be used for inter-frame FFT, or that Ring-FIFO storage must be used, etc., which will not be repeated here.

[0115] In addition, as Figure 1 As shown, if the user wants to obtain a more intuitive and accurate result, in some embodiments, Fig.13 As shown, after obtaining the target, the following steps are also performed:

[0116] Step 205 , verifying the targets in each preset area according to the targets falling into the preset areas, where the preset areas are obtained by dividing the detection space.

[0117] Step 206: output the detection results of each preset area according to the verified targets.

[0118] That is to say, taking into account that for a target with a certain volume and occupying a certain space, multiple destination points are usually detected, and the currently detected target is used to continue to be verified in combination with the area division to improve the detection result. Based on the division of the detection space in the application scenario, the detection results of each preset area are output, so that the target will be presented in the preset area. The distribution is more intuitive and accurate, which is more conducive to users making decisions based on the output results and providing a better user experience.

[0119] To facilitate better understanding of those skilled in the art Fig.13 The embodiment shown, the steps of which will be explained below.

[0120] In step 205, the detection space and the preset area are not limited, and they may be different according to different application scenarios, application requirements, etc. For example, in the application scenario of the vehicle-mounted radar, the detection space may be inside the vehicle, and the preset area at this time includes at least any one of the following items, including the area corresponding to the seat and the area corresponding to the aisle (foot mat), so that the driver or adults and other objects can better perceive the situation inside the vehicle. For example, in a factory work scenario, the detection space may be a factory building, and the preset area may include the workstations of each employee to avoid production risks caused by employees not being able to monitor the working conditions of the machines due to employees no longer being at their workstations. They will not be described one by one here. In the following, for ease of understanding, the above-mentioned in-vehicle space will be used as an example for explanation, but this does not mean that the corresponding solution can only be implemented in the car.

[0121] Taking the interior of a car as an example, the interior space is abstracted as follows: Fig.14 The coordinate system shown in FIG. 1 is a coordinate system in which the horizontal axis of the coordinate system represents the azimuth angle relative to the radar installed in the car, and the horizontal axis of the coordinate system represents the elevation angle relative to the radar installed in the car. At this time, the car is divided into six preset areas, namely, three rear seats and three aisles in front of three seats. Fig.14 Different filling areas are shown. According to different requirements, such as Fig.14 As shown, different preset areas may overlap or not overlap with each other; certain areas in the vehicle may belong to multiple preset areas at the same time or may not belong to any area.

[0122] It should be noted that Fig.14 This is only an abstract way of detecting space. In some embodiments, the detection area can be abstracted from all or one of the dimensions of distance, azimuth, and pitch, or all or one of the dimensions of xyz along the rectangular coordinate system, and further divided into preset areas, which will not be described in detail here.

[0123] In step 205, the verification method is not limited, for example, the verification may be performed based on the number or proportion of target points.

[0124] For example, in some embodiments, Fig.15 As shown, according to the targets falling into each preset area, verifying the targets in each preset area can be achieved through the following steps:

[0125] Step 2051 , verifying the targets in each preset area according to the proportion of the targets falling in each preset area among the detected targets.

[0126] For example, in some embodiments, Fig.16 As shown, according to the targets falling into each preset area, verifying the targets in each preset area can be achieved through the following steps:

[0127] Step 2052: verify the targets in each preset area according to the signal-to-noise ratio of the targets falling within each preset area.

[0128] Of course, the above are only examples. In some cases, targets at other locations can also be verified based on the target point falling into locations such as the driver's seat where targets are known to exist. I will not go into details here.

[0129] In step 206, the output method is not limited. The detection results of each preset area can be directly output as a point cloud, or the divided preset areas and the point clouds of each preset area can be simultaneously output, etc., which will not be listed here one by one.

[0130] In order to facilitate those skilled in the art to better understand the above embodiment, the following will be described by taking proportion verification as an example.

[0131] Assume that 11 target points are detected in a target detection process, and the preset area is divided as follows: Fig.14 As shown, the distribution of target points in the preset area is shown as Fig.17 As shown, the area seats A (6 target points), seat B (2 target points), seat C (0 target points), aisle A (0 target points), aisle B (1 target point) and aisle C (0 target points), where, Fig.17 The mid-target point is represented by a solid circle.

[0132] At this time, the algorithm obtained according to the above embodiment will be as follows:

[0133] Initialize the flags flag_region_i of the six regions to be 0, i.e., no one exists (i=0, 1, ..., 5);

[0134] If the total valid target number total_valid_tgt_num is 0, it is determined that there is no one in the car and the process ends, otherwise the following operations are performed;

[0135] Traverse all regions. For region i, if the statistical value of the number of valid targets belonging to the region tgt_num_region_i exceeds 25% of the total number of valid targets total_valid_tgt_num, set the flag flag_region_i of region i to 1;

[0136] If there is a region, if its flag_region_i is 1, then set flag_region_empty to 0, that is, there is someone in the car;

[0137] Traverse the seat area, if there is any seat area where flag_region_i is 1, then determine that the corresponding aisle area is empty (for example, if it is determined that there is someone in seat A, then determine that there is no one in aisle A);

[0138] Output the result.

[0139] At this time, the output will be as follows Fig.16 As shown, area seat A displays 6 target points, and the rest of the preset areas are empty.

[0140] It should be noted that the threshold value of 25% in the above example is only an example. In other embodiments, other threshold values ​​or other judgment conditions may be used, which will not be described in detail here.

[0141] It is also understandable that, during the detection process, it may be necessary to combine different periodic motions for observation, so that the target detection result can be obtained more accurately. Based on this, in some embodiments, such as Fig.18 As shown, the target detection method includes the following steps:

[0142] Step 1801, perform distance dimension FFT on the frame data.

[0143] Step 1802, extracting target data from the frame data obtained after range-dimensional FFT processing.

[0144] Step 1803, windowing the target data according to the sliding windows corresponding to the cycles of different vital sign parameters of the living body, and performing digital signal processing on the windowed data corresponding to sliding windows of different lengths.

[0145] The periods of the vital sign parameters corresponding to different sliding windows are different, and the length of each sliding window has the same order of magnitude as the period of the vital sign parameter.

[0146] Among them, step 1801 and step 1802 are roughly the same as the aforementioned step 201 and step 202, and the only difference is that the subsequent step 1803 and step 203 are based on multiple different periodic motions for detection, while the other describes only one. It can be understood that when detection is based on multiple different periodic motions, after the accumulation of data with a shorter period is completed, the data continues to accumulate until the accumulation of data with a longer period is completed, and the processing of the data is completed. For example, assuming that one periodic motion corresponds to 32 frames and one periodic motion corresponds to 128 frames, both of which need to be observed, then in the process of target detection, a detection result is output from the time the target data corresponding to the 32nd frame data is acquired, and a new detection result continues to be output after the target data corresponding to the 33rd frame data is acquired...until the target data corresponding to the 128th frame data is acquired, two detection results will be output (one is the detection result corresponding to the 32 frame data, and the other is the detection result corresponding to the 128 frame data), or, the result after the two results are comprehensively processed.

[0147] In this way, by obtaining the period of the vital sign parameters of the living body, the length of the sliding window can be adjusted in time, so as to better detect the living target and better monitor the status of the living body (such as children, pets, etc.) in order to monitor and take corresponding measures to the performance of the living body in time.

[0148] It should be noted that the embodiments of the present application do not limit the vital sign parameters. For example, the vital sign parameters may include breathing. Another example is that the vital sign parameters may include heartbeat and / or pulse. Of course, the above are only examples. Under other requirements, other vital sign parameters that can be reflected as motion characteristics can also be used, which will not be listed here one by one.

[0149] Thus, in the above-mentioned embodiment, the frame-level FFT is used to replace the traditional chirp-level Doppler FFT to realize the effective accumulation of the target's motion energy so as to achieve the purpose of accurate detection and measurement, and the result update frequency can be further improved by performing sliding window FFT processing on multiple frames; by performing post-processing methods such as regional statistics on the target points detected by the radar, it is possible to determine whether there are people in each area.

[0150] Specifically, in the above-mentioned embodiment, the speed of the target is determined by replacing the traditional chirp-level Doppler dimension FFT to achieve accurate detection of the target of interest, and the update frequency of the result can be further improved by performing sliding window FFT processing on multiple frames, thereby improving the real-time performance of the system. At the same time, the amount of data processed can be reduced by processing only the distance and / or Doppler area of ​​interest. In addition, when estimating the background noise, the background noise is obtained by taking the minimum value based on the estimation of the current distance unit (bin) combined with the estimation of the last distance unit of interest, which can be more suitable for application scenarios in relatively sealed environments such as in the cabin of a car. When judging the regional logic, by counting the target points detected in each frame according to the preset area design, and judging the counting results according to the preset rules, it is possible to accurately judge whether there is a target in each preset area.

[0151] Fig.19 Schematic diagram of the process of realizing target detection based on per-channel in the embodiment of the present application. Fig.19 As shown, based on Figure 1 Based on the flow structure shown in FIG. 1 , a scheme for performing incoherent accumulation (or other channel-to-channel accumulation) of multi-channel RD spectra into single-channel data for CFAR is proposed. In this embodiment, CFAR processing is performed on all channels one by one, and then binary integration in channel-domain is performed, and then the horizontal / azimuth and elevation angles are estimated. Specifically, for a system with M channels, each channel performs the following steps: Figure 1 After the inter-frame FFT shown in , CFAR processing is performed on each channel to obtain M bit masks, each element of which can be of bool type, and each range unit (range bin) and / or Doppler bin is judged to determine whether there is a target (if it is 1, it is defined as present, and if it is 0, it is defined as not present). At the same time, these M bit masks can be accumulated, and the range of each element can be defined as 0 to (M-1). Subsequently, the accumulated mask can be subjected to secondary detection, that is, each element of the accumulated mask can be compared with a preset threshold value M0. If the value x of the unit to be detected (such as an element of the accumulated mask) is i >M0, it can be output that there is a target in the unit to be detected. If x i ≤M0, it can be output that there is no target in the unit to be detected. i , M, M0 are positive integers.

[0152] exist Fig.19In the detection method flow shown, since each channel is subjected to CFAR processing separately, the performance loss caused by channel imbalance can be effectively reduced, and the detection method has better robustness in practical applications.

[0153] Fig. 20 FIG. 1 is a flow chart of target detection based on frame-level FFT combined with DAE CFAR in an embodiment of the present application. Fig. 20 As shown, in Figure 1 Based on the process structure shown, based on the frame-dimensional FFT (frame-FFT), DAE CFAR (Constant False Alarm Rate) is used to estimate the azimuth and elevation angles for CFAR data. Specifically, after Doppler dimension CFAR detection is performed on the inter-frame FFT data, azimuth dimension DBF (Azimuth DBF) is performed, and the azimuth CFAR processing (Azimuth CFAR) is continued in combination with the NVE (noise variance estimator) noise floor estimation; similarly, the elevation DBF (Elevation DBF) is performed on the azimuth CFAR data, and the elevation CFAR processing (Elevation CFAR) is continued in combination with the NVE noise floor estimation, thereby obtaining the azimuth and elevation information of the target.

[0154] Fig.21 Based on Fig. 20 The schematic diagram of the azimuth or elevation CFAR in the target detection process is shown in the figure. Fig.21 As shown, the azimuth / elevation power data can be calculated by the NVE engine in a HIST manner, and the azimuth / elevation DBF noise floor estimation can be performed by selecting a rank value (such as mean, median, etc.). When performing DBF, all peaks in the DBF spectrum can be obtained, and the SNR of all peak points and the estimated noise floor can be used to determine whether to output as a peak.

[0155] For example, when performing DBF, the amplitudes of all peak points in the DBF spectrum, including the global maximum (Global max) peak and the local maximum (Local max) peak, can be obtained; for any peak, if its SNR exceeds a predetermined threshold, and the difference between its amplitude (local max) and the global maximum amplitude (Global max) is within a preset range, and the number of peaks that have been output is less than the preset number, then the peak is output, that is, output as the real target point.

[0156] Fig. 22Schematic diagram of the process of post-processing the target in the embodiment of the present application. Fig. 22 As shown, it can be Figure 1 On the basis of the process structure shown, after the azimuth / elevation angle estimation, the output target point cloud data can be processed in combination with the machine learning model to accurately detect the target in the area of ​​interest. Specifically, for the single target point cloud data obtained after the azimuth / elevation angle estimation, the false alarm suppression technology of ML (Machine Learning, such as SVM or RF algorithm) can be combined to suppress non-ideal data such as false alarms generated by noise, stationary clutter or target multipath, that is, for the target point cloud data such as the pitch DBF&DoA output, continue to perform operations such as ML-based false-alarm suppression, clustering and / or ML-based target classification, where the clustering process can be agglomerative clustering or DBSCAN algorithms. At the same time, the point cloud data after clustering processing can also be input into a trained machine learning model (such as SVM or RF) to determine whether there is a person in the current processing. If there is, its location can be further determined, as well as the distinction and judgment of physical targets such as adults, children, and infants.

[0157] Fig. 22 In the embodiment shown, the influence of non-ideal factors such as noise, stationary clutter and target multipath can be effectively reduced, the difficulty of selecting regional parameters and the difficulty of complex regional judgment logic design can be effectively reduced, and point cloud information can be more effectively utilized, especially for applications in complex scenes, to obtain better target detection and distinction judgment performance.

[0158] Fig.23 This is a flow chart of post-processing the target in combination with the DL algorithm in the embodiment of the present application. Fig.23 As shown, it can be Figure 1 Based on the process shown, after extracting the data obtained from at least one of the operation steps such as multi-frame accumulation (store frame), frame Fourier transform (FrameFFT), non-coherent integration (Non-coherent integration) (or other channel accumulation), constant false alarm detection (CFAR), etc., a deep learning algorithm (Deep Learning, referred to as DL) is used to judge, locate and identify the target, for example, the identification of whether the physical target is an adult or a child (Adult / Child Classification).

[0159] Specifically, the subsequent target differentiation and judgment can be achieved by using 1D-FFT raw data as DL input (Access 1D-FFT data as raw input of DL), using 2D-FFT raw data between frames as DL input (Access 2D-FFT data as raw input of DL), using non-coherent integrated 2DFFT data (or other inter-channel accumulation) as DL input (Access non-coherent integrated 2DFFT data as raw input of DL) and / or using SNR data as DL input (Access SNR data as raw input of DL). The raw data here can be complex data, modulus value of complex data, modulus square of complex data or other transformation forms, and can be values ​​in the linear domain, values ​​in the dB (log) domain or other calculation forms.

[0160] based on Fig.23 The process shown in the figure uses at least one of the above methods to extract multi-frame 1D-FFT data, multi-channel data after multi-frame FFT, and energy (power) or SNR data after multi-frame FFT and SISO-combine, and then inputs the extracted data into the constructed deep learning model (such as CNN or Transformer, etc.) to determine whether there is a target in the current processing, and to determine the current target type (such as determining whether the target is an adult or a child, etc.).

[0161] It should be noted that Fig.23 The process steps and related descriptions shown in Fig. 22 The process steps shown and the related descriptions may be compatible with each other; for example, Fig.23 The Region HIST and subsequent modules in the process shown can be used Fig. 22 In the process shown, the ML-based false alarm suppression and its subsequent modules are partially or completely replaced with each other.

[0162] Fig.24 FIG. 1 is a flow chart of a target detection method combined with a DL algorithm in an embodiment of the present application. Fig.24 As shown, it can be Figure 1On the basis of the process shown, after the inter-frame FFT (Frame FFT) processing, two-dimensional digital beamforming (2D-DBF) processing and DL-based target classification processing (DL-based Classification) operations are directly performed. That is, after the inter-frame FFT, CFAR processing is not performed, but 2D-DBF operations are performed on each region of interest (or unit interval) in the azimuth and elevation dimensions to obtain a number of RD spectra corresponding to the region of interest; subsequently, the above-mentioned RD spectra are input into the constructed deep learning model (DL) for target judgment. In some optional embodiments, the relationship between the above-mentioned region of interest and the RD spectra can also be one-to-many, that is, at least two RD spectra can be obtained based on one region of interest, and the specific number can be adjusted according to actual needs.

[0163] The above DL input can be linear or dB domain amplitude or power, and normalization processing and other operations can also be performed. At the same time, the deep learning model constructed above can be classified according to the input, and the categories distinguished can be: whether there is a target, the attributes of the target, the specific unit interval location (i.e., regional location) of each target, etc.

[0164] For example, in the application scenario where the rear area in the car is the area of ​​interest or the target area, and it is divided into 3 seat areas (interval units) and corresponding 3 aisle areas, after the inter-frame FFT, the center positions of the above 6 areas of interest (the areas of interest are also set as 3 seat areas) can be subjected to azimuth and elevation 2D-DBF to obtain 6 RD spectra (or 3 RD spectra) corresponding to the areas of interest, and at least part of the above 6 RD spectra (or 3 RD spectra) are input into the constructed deep learning model for target judgment and differentiation. For example, the constructed deep learning model performs classification based on the input to determine whether there is a living target in the above area of ​​interest. If there is a living target, it can be further determined whether the living target is an adult, a child, an infant or a pet, etc., and can further determine the specific location information of the living target, such as which seat or aisle area the living target is in.

[0165] Fig.24 In the embodiment shown, the target detection method is a processing flow based on dual drive of model data, which has low requirements for signal processing and can effectively avoid the steps of selecting signal processing parameters, regional parameters, designing CFAR logic and regional judgment logic, thereby greatly reducing the difficulty of solution implementation and design.

[0166] In some optional embodiments, for Figure 1 , Figure 19-Figure 24In the target detection method shown, all FFT processing can be windowed FFT, and can also be implemented by using SVA combined with FFT.

[0167] Fig.25 FIG. 1 is a flow chart of another target detection method combined with a DL algorithm in an embodiment of the present application. Fig.25 As shown, it can be Fig.24 Based on the process shown, after the range Fourier transform (Range FFT, also called 1D-FFT), 2D-DBF is performed first, and then frame data accumulation and inter-frame FFT processing are performed. That is, before the inter-frame FFT, the azimuth and elevation angles of the center position of the region of interest (such as the three-seat area) are firstly performed 2D-DBF. This embodiment can realize the solution of parallel processing of data processing and wave transmission time, thereby effectively reducing the processing time.

[0168] In some optional embodiments, for Figure 1 , Figure 19-Figure 25 In the target detection method shown, the multi-frame sliding window FFT can also replace other time-frequency transform processing, such as short-time Fourier transform, fractional Fourier transform, etc., which can also achieve effective detection of the target of interest.

[0169] In some optional embodiments, for Figure 1 , Figure 19-Figure 25 In the target detection method shown, the multi-frame sliding window FFT can also be replaced by a high-pass FIR, a Comb FIR, or a filter optimized and synthesized according to an optimization function, so as to achieve the purpose of obtaining similar or even higher performance of multi-frame FFT processing with less hardware resources.

[0170] like Figure 24-25 As shown, by combining 2D-FFT with DL recognition (DL-based Classification), the accuracy of target recognition can be effectively improved. At the same time, the 2D-FFT steps and the DL recognition steps can be flexibly set between various steps in signal processing according to actual needs, such as setting the 2D-FFT step after Range FFT, or after Frame FFT, etc.

[0171] Fig.26 FIG. 1 is a flow chart of another target detection method combined with a DL algorithm in an embodiment of the present application. Fig.26 As shown, it can be Fig.24 or Fig.25On the basis of the process shown, that is, after the range-dimensional Fourier transform (Range FFT, also called 1D-FFT), the frame data is accumulated after 2D-DBF, and the DL-based target classification operation (ComplexDL-based Classification) is directly performed. That is, in the entire target detection process, no inter-frame FFT processing operation is performed, and 2D-DBF is directly performed on the center position of the area of ​​interest (such as the 3 seat area) after 1D-FFT to obtain the corresponding number of distance-frame maps (such as 3 range-frames), and then the above distance-frame maps are input into the constructed (for example, complex) deep learning model for target differentiation and judgment and other operations.

[0172] exist Fig.26 In the embodiment shown, since the FFT processing step between frames is avoided, the amount of calculation can be effectively reduced; at the same time, if a complex deep learning model is adopted, a higher processing gain can be obtained compared to FFT, thereby achieving the purpose of improving target detection performance; in addition, this embodiment also effectively shortens the processing level, which can facilitate its efficient scheduling operations, etc.

[0173] In some optional embodiments, for Figure 24-26 The DBF in the target detection method shown can be replaced by Capon, MUSIC, ESPRINT and other algorithms and their variants. At the same time, the beam synthesis at the center of the area of ​​interest and / or the antenna arrangement can be optimized and integrated to further improve the system target detection performance.

[0174] It should be noted that the embodiments of the present application can refer to each other and be compatible without conflict, and the order and whether to set each functional module can be adjusted according to the needs. At the same time, for a system with a baseband module and a main control module, when the system detects a target by transmitting electromagnetic waves and receiving corresponding echo signals, each step in each target detection method of the embodiment of the present application can be configured to run in the baseband module and / or the main control module according to actual needs and the data processing capacity and timeliness of the system operation. The relevant examples in the figure can be used as a reference for some of the options.

[0175] In some optional embodiments, Figure 1 , Figure 19-Figure 26 The target detection process in the above is applicable to TDM-MIMO systems. For systems that do not use TDM-MIMO, different phase shifts can be preset when different Tx are transmitting. After the waveform of the directly transmitted waveform is digitally synthesized (phased array), the target detection process can be combined with the phase shift of the directly transmitted waveform to obtain the target detection result. Figure 24-26 The processing flow shown in the figure performs operations such as target detection and judgment.

[0176] In some other embodiments, for systems using a pulse system, a pulse compression system or an ultra-wideband system, directly using Figure 24-26 The method flow shown is used to detect targets, and the performance of target detection can be further improved by adopting the digital synthesis technology of the transmit beam. Among them, for the system of pulse compression system, 1D-FFT processing is not required, but the step of pulse compression needs to be added.

[0177] It should be noted that the above-mentioned processing methods corresponding to systems of different standards can also be applied to systems of mixed standards. For example, for a system with 6 transmitting antennas (i.e., 6 Tx), 3 of the transmitting antennas (3 Tx) can be TDM standards, and the other 3 transmitting antennas can be other standards. In the subsequent processing, the processing method corresponding to the above-mentioned standards can be used.

[0178] After the inter-frame FFT (Frame FFT) processing, two-dimensional digital beamforming (2D-DBF) processing and DL-based target classification processing (DL-based Classification) operations are directly performed. That is, after the inter-frame FFT, CFAR processing is not performed, but 2D-DBF operations are performed on the azimuth and elevation angles of each region of interest (or unit interval) to obtain the RD spectrum corresponding to the number of regions of interest; subsequently, the above-mentioned RD spectrum is input into the constructed deep learning model (DL) for target judgment. In some optional embodiments, if the inter-frame FFT method is not used, but FIR, STFT (Short-Time Fourier Transform) and other methods are used for processing, the above-mentioned RD spectrum corresponds to FIR as a one-dimensional spectrum output, and the corresponding STFT is a three-dimensional spectrum for time-frequency transformation.

[0179] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0180] The embodiment of the present application also provides an integrated circuit, which may include a radio frequency module, an analog signal processing module and a digital signal processing module connected in sequence; the radio frequency module is used to generate a radio frequency transmission signal and receive an echo signal; the analog signal processing module is used to down-convert the echo signal to obtain an intermediate frequency signal; the digital processing module is used to perform analog-to-digital conversion on the intermediate frequency signal to obtain a digital signal; and based on the target detection method in the embodiment of the present application, the digital signal is processed to achieve the purpose of target detection, for example, the integrated circuit may be a millimeter wave radar chip (chip or die). Among them, the above-mentioned digital processing module may include a baseband module and a main control module, etc., and each module may be configured to execute each corresponding step in the target detection method in the above-mentioned embodiment.

[0181] In some embodiments, the digital signal processing module is used to perform distance-dimensional FFT on the frame data obtained by analog-to-digital conversion, extract target data from the frame data obtained after distance-dimensional FFT processing, window the target data according to a sliding window, and perform digital signal processing on the windowed data; wherein the length of the sliding window corresponds to a duration of the same order of magnitude as the period of the target's periodic motion.

[0182] In some embodiments, the digital signal processing module includes a baseband unit and a main control unit; the baseband unit is connected to the analog signal processing module and the main control unit respectively, and is used to perform distance-dimensional FFT on the frame data, and extract target data from the frame data obtained after the distance-dimensional FFT processing; the main control unit is used to obtain the target data extracted by the baseband unit, window the target data according to the sliding window, and perform digital signal processing on the windowed data.

[0183] In some embodiments, the digital signal processing module is also used to verify the targets in each preset area after obtaining the target according to the targets falling into each preset area, and the preset areas are obtained by dividing the detection space; according to the verified targets, the detection results of each preset area are output.

[0184] In some embodiments, the digital signal processing module is further used to perform distance dimension FFT processing on at least two frames of data in the echo signal, and then continue Doppler FFT processing between frames to obtain the period of the vital sign parameters of the living body.

[0185] In some optional embodiments, the integrated circuit may be an AiP (Antenna-In-Package) chip structure, an AoP (Antenna-On-Package) chip structure, or an AoC (Antenna-On-Chip) chip structure.

[0186] According to other embodiments of the present application, an electromagnetic wave sensor is also proposed. The electromagnetic wave sensor may include an antenna, and an integrated circuit as described above. The integrated circuit is electrically connected to the antenna for receiving and transmitting electromagnetic wave signals. For example, the electromagnetic wave sensor may include: a carrier, an integrated circuit and an antenna as described in any of the above embodiments, and the integrated circuit may be arranged on the carrier; the antenna may be arranged on the carrier, or the integrated circuit may be integrated with the integrated circuit as an integrated device arranged on the carrier (that is, the antenna may be an antenna provided in an AiP, AoP or AoC structure at this time); the integrated circuit is connected to the antenna (that is, the sensor chip or integrated circuit is not integrated with an antenna at this time, such as a conventional SoC, etc.), for receiving and transmitting electromagnetic wave signals. The carrier may be a printed circuit board PCB, and the corresponding transmission line may be a PCB trace.

[0187] It should be noted that the electromagnetic waves in the present application may include radio waves and light waves. Radio waves include short waves, medium waves, long waves and microwaves, etc. Microwaves include centimeter waves (i.e. electromagnetic waves in the range of 3GHz to 30GHz, such as electromagnetic waves in the range of 3.1GHz to 10.6GHz, electromagnetic waves in the frequency band of 24GHz, etc.) and millimeter waves (i.e. electromagnetic waves in the range of 30GHz to 300GHz, such as electromagnetic waves in the frequency band of 60GHz, electromagnetic waves in the frequency band of 77GHz (such as 77GHz to 81GHz, etc.)), and light waves may include ultraviolet rays, visible light, infrared rays, and lasers, etc., among which the electromagnetic wave frequency band of lasers is (3.846 to 7.895)*10^5GHz, i.e., lasers are included in part of the frequency bands of ultraviolet rays and visible light.

[0188] An embodiment of the present application provides a device, which may include: a device body; and an electromagnetic wave sensor as described above, which is arranged on the device body; wherein the electromagnetic wave sensor is used for target detection and / or communication to provide reference information for the operation of the device body.

[0189] The embodiment of the present application also provides an electronic device, which can be expressed in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit, at least one storage unit, a bus connecting different system components (including a storage unit and a processing unit), a display unit, etc. Among them, the storage unit stores a program code, and the program code can be executed by the processing unit so that the processing unit executes the method described in this specification according to various exemplary embodiments of the present application. The storage unit may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) and / or a cache memory unit, and may further include a read-only memory unit (ROM).

[0190] The storage unit may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0191] The bus may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0192] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface. In addition, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter may communicate with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0193] For example, the electronic device in the embodiment of the present application may also include: a device body; and an electromagnetic wave sensor as described in any of the above embodiments arranged on the device body; wherein the electromagnetic wave sensor can be used to realize functions such as target detection and / or wireless communication.

[0194] Specifically, based on the above embodiments, in an optional embodiment of the present application, the electromagnetic wave sensor can be arranged outside the device body, or arranged inside the device body, and in other optional embodiments of the present application, the electromagnetic wave sensor can also be arranged partially inside the device body and partially outside the device body. The embodiments of the present application are not limited to this, and the specific situation can be determined.

[0195] In an optional embodiment, the device body can be a component or product used in fields such as smart cities, smart homes, transportation, smart homes, consumer electronics, security monitoring, industrial automation, in-cabin detection (such as smart cockpits), medical devices, and health care. For example, the device body can be intelligent transportation equipment (such as cars, bicycles, motorcycles, ships, subways, trains, etc.), security equipment (such as cameras), liquid level / flow rate detection equipment, smart wearable devices (such as bracelets, glasses, etc.), smart home devices (such as sweeping robots, door locks, televisions, air conditioners, smart lights, etc.), various communication devices (such as mobile phones, tablet computers, etc.), and gates, smart traffic lights, smart signs, traffic cameras, and various industrialized robotic arms (or robots), etc. It can also be various instruments for detecting life characteristic parameters and various devices equipped with the instruments, such as life characteristic detection in car cabins, indoor personnel monitoring, smart medical equipment, consumer electronic equipment, etc.

[0196] The embodiment of the present application also provides a non-transitory computer-readable storage medium on which computer-readable instructions are stored. When the instructions are executed by a processor, the processor executes the feeder unequal length compensation method as described above.

[0197] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. The technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the implementation method of the present application.

[0198] The software product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0199] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0200] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0201] The computer-readable medium carries one or more programs. When the one or more programs are executed by a device, the computer-readable medium implements the aforementioned functions.

[0202] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0203] According to an embodiment of the present application, a computer program is proposed, including a computer program or an instruction, which, when executed by a processor, can execute the method described above. In an optional embodiment, the above-mentioned integrated circuit can be a millimeter wave radar chip. The type of digital function module in the integrated circuit can be determined according to actual needs. For example, in a millimeter wave radar chip, the data processing module can be used for functions such as distance-dimensional Doppler transform, velocity-dimensional Doppler transform, constant false alarm detection, wave direction detection, point cloud processing, etc., to obtain information such as the distance, angle, speed, height, micro-Doppler motion characteristics, shape, size, surface roughness and dielectric properties of the target.

[0204] It should be noted that radio devices can achieve functions such as target detection and / or communication by transmitting and receiving radio signals to provide the device body with detection target information and / or communication information, thereby assisting or even controlling the operation of the device body.

[0205] For example, when the above-mentioned device body is applied to an advanced driver assistance system (i.e., ADAS), the radio device (such as millimeter-wave radar) as a vehicle-mounted sensor can assist the ADAS system to realize application scenarios such as adaptive cruise control, automatic braking assistance (i.e., AEB), blind spot detection warning (i.e., BSD), assisted lane change warning (i.e., LCA), reversing assistance warning (i.e., RCTA), parking assistance, rear vehicle warning, collision avoidance, pedestrian detection, and in-cabin liveness detection (i.e., CPD).

[0206] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] The above-described embodiments only express the preferred embodiments of the present invention and the technical principles used. The description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It is possible for those skilled in the art to make various obvious changes, readjustments and substitutions without departing from the protection scope of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, and the protection scope of the patent of the present invention is determined by the scope of the attached claims.

Claims

1. A target detection method, characterized in that: include: Perform distance dimension FFT on frame data; Extract target data from the frame data obtained after distance dimension FFT processing; Performing sliding windowing on the target data, and performing digital signal processing on the windowed data to obtain target information; The length of the sliding window corresponds to a duration of the same order of magnitude as the period of the periodic motion of the target.

2. The target detection method according to claim 1, characterized in that: The step of extracting target data from the frame data obtained after the distance dimension FFT processing comprises: At least part of the data or at least part of the data processed is extracted from each frame of data obtained after the distance dimension FFT processing as the target data.

3. The target detection method according to claim 1, characterized in that: The step of extracting target data from the frame data obtained after the distance dimension FFT processing comprises: According to each frame of data obtained after the distance dimension FFT processing, the parameters corresponding to each distance unit in each frame of data are determined to generate the target data.

4. The target detection method according to any one of claims 1 to 3, characterized in that: The step of performing digital signal processing on the windowed data to obtain target information includes: Performing noise floor estimation within a preset range, wherein an upper limit of the preset range is determined according to a noise floor estimation result of a distance unit with the longest distance; Based on the estimated noise floor, constant false alarm detection is performed to obtain target information.

5. The target detection method according to claim 4, characterized in that: The noise floor estimation within the preset range is implemented by the following expression: n i =min(n i α*n g ): Among them, n i ' is the noise floor estimation result of the ith distance unit within the preset range, n i is the initial noise floor estimation result of the ith distance unit, α is the preset parameter, α≥1, n g is the initial noise floor estimation result of the farthest distance unit.

6. The target detection method according to any one of claims 1 to 3, characterized in that: The step of extracting target data from the frame data obtained after the distance dimension FFT processing comprises: Extracting data within a preset distance range from the frame data obtained after distance dimension FFT processing as the target data; and / or, The step of performing digital signal processing on the windowed data includes: Digital signal processing is performed on the data within the preset Doppler range in the windowed data.

7. The target detection method according to any one of claims 1 to 3, characterized in that: In the case where the data is cached using RingFIFO, the method further includes: Data is read from the Ring FIFO, and the read data is rearranged.

8. A target detection method, characterized in that: include: Perform target detection according to the target detection method according to any one of claims 1 to 7 to obtain a target; According to the target falling into each preset area, verifying the target in each preset area, wherein the preset area is obtained by dividing the detection space; According to the verified target, the detection result of each preset area is output.

9. The target detection method according to claim 8, characterized in that: The verifying the target in each preset area according to the target falling in each preset area includes: The targets in each preset area are verified according to the proportion of the targets falling within each preset area in the detected targets.

10. The target detection method according to claim 8, characterized in that: The verifying the target in each preset area according to the target falling in each preset area includes: The target in each preset area is verified according to the signal-to-noise ratio of the target falling within each preset area.

11. The target detection method according to any one of claims 8 to 10, characterized in that: The detection space is inside the vehicle, and the preset area includes at least any one of the following: an area corresponding to a seat and an area corresponding to an aisle.

12. A method for target detection, characterized in that: include: Perform distance dimension FFT on frame data; Extract target data from the frame data obtained after distance dimension FFT processing; Windowing the target data according to the sliding windows corresponding to the cycles of different vital sign parameters of the living body, and performing digital signal processing on the windowed data corresponding to the sliding windows of different lengths respectively; The periods of the vital sign parameters corresponding to different sliding windows are different, and the length of each sliding window has a duration of the same order of magnitude as the period of the vital sign parameter.

13. The target detection method according to claim 12, characterized in that: The vital sign parameters include breathing.

14. The target detection method according to claim 12, characterized in that: The vital sign parameters include heartbeat and / or pulse.

15. An integrated circuit, characterized in that: It includes a radio frequency module, an analog signal processing module and a digital signal processing module connected in sequence; Wherein, the RF module is used to generate RF transmission signals and receive RF reception signals; The analog signal processing module is used to down-convert the RF received signal to obtain an intermediate frequency signal; The digital signal processing module is used to perform analog-to-digital conversion on the intermediate frequency signal, perform distance dimension FFT on the frame data obtained by the analog-to-digital conversion, extract target data from the frame data obtained after the distance dimension FFT processing, window the target data according to the sliding window, and perform digital signal processing on the windowed data; The length of the sliding window corresponds to a duration of the same order of magnitude as the period of the periodic motion of the target.

16. The integrated circuit according to claim 15, characterized in that The digital signal processing module includes a baseband unit and a main control unit; The baseband unit is connected to the analog signal processing module and the main control unit respectively, and is used to perform distance dimension FFT on the frame data, and extract target data from the frame data obtained after the distance dimension FFT processing; The main control unit is used to obtain the target data extracted by the baseband unit, window the target data according to a sliding window, and perform digital signal processing on the windowed data.

17. The integrated circuit according to claim 15 or 16, characterized in that: The digital signal processing module is also used to verify the target in each preset area according to the target falling into each preset area after obtaining the target, and the preset area is obtained by dividing the detection space; According to the verified target, the detection result of each preset area is output.

18. The integrated circuit according to claim 15 or 16, characterized in that: The digital signal processing module is also used to perform distance dimension FFT processing on at least two frames of data in the echo signal, and then continue Doppler FFT processing between frames to obtain the period of the vital sign parameters of the living body.

19. An electromagnetic wave sensor, characterized in that: include: Carrier; An integrated circuit as claimed in any one of claims 15 to 18, arranged on a carrier; An antenna is arranged on the carrier, or the antenna and the integrated circuit are integrated into one device and arranged on the carrier; The integrated circuit is connected to the antenna and is used to transmit a radio frequency transmission signal and / or receive a radio frequency reception signal.

20. A device, characterized in that include: Equipment body; and an electromagnetic wave sensor as claimed in claim 19 disposed on the device body; The electromagnetic wave sensor is used for target detection and / or communication to provide reference information for the operation of the device body.

Images