Target behavior recognition method and device based on space-time-frequency and image-domain micro-Doppler features

Abstract

The present application relates to a method and device for target behavior recognition based on space-time-frequency and image-domain micro-Doppler features. The method first generates a time-domain signal of the target area based on the radar echo signal of the target area; then generates a two-dimensional image of the target area based on the time-domain signal, and determines the two-dimensional spatial position of at least one object; then, based on the two-dimensional spatial position, performs single-point three-dimensional imaging of multiple height points on at least one object at different height points to obtain a single-point three-dimensional imaging result; finally, based on the single-point three-dimensional imaging results of multiple height points in a continuous time, a micro-Doppler spectrum of at least one object is generated, and the behavior of at least one object is identified based on the micro-Doppler spectrum. The method combines space, time, and frequency features through a space-time-frequency collaborative framework, thereby improving the recognition accuracy of object behavior and enabling efficient multi-object behavior recognition in complex scenes.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of radar signal processing technology, and in particular to a method and device for target behavior recognition based on space-time-frequency and image-domain micro-Doppler features. Background Art

[0002] Compared to traditional visual and inertial sensors, radar technology offers many unique advantages, such as high resolution, strong penetration, and good privacy protection. Radar technology can capture richer target information and provide multi-dimensional behavioral data, which is crucial for accurately identifying human behavioral patterns and characteristics. Radar technology identifies dynamic human behavior by precisely capturing the Doppler effect caused by tiny movements on or within the human body. Behavior recognition based on the micro-Doppler effect has become an important research direction in the field of human behavior perception.

[0003] Although behavior recognition based on micro-Doppler features has made some progress in single-person behavior recognition, in complex scenarios, the micro-Doppler signals of multiple targets may overlap with each other, resulting in signal interference and thus reducing recognition accuracy. Summary of the Invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to propose a target behavior recognition method and device based on space-time-frequency and image domain micro-Doppler features, which can combine spatial, temporal and frequency features to improve the behavior recognition accuracy and system robustness of at least one object.

[0006] A first aspect of the embodiments of the present application provides a target behavior recognition method for space-time-frequency coordinated processing, the method comprising:

[0007] generating a time domain signal of the target area based on a radar echo signal of the target area; wherein the target area includes at least one object;

[0008] generating a two-dimensional image of the target area according to the time domain signal, and determining a two-dimensional spatial position of the at least one object from the two-dimensional image;

[0009] performing single-point three-dimensional imaging of the at least one object at multiple height points at different height points according to the two-dimensional spatial position to obtain single-point three-dimensional imaging results at different height points;

[0010] generating time series data based on the single-point three-dimensional imaging results of any height point of any object in a continuous time, and extracting an image-domain micro-Doppler spectrum including amplitude information and phase information of the at least one object from the time series data based on a discrete short-time Fourier transform;

[0011] The behavior of the object in the target area is identified according to the image-domain micro-Doppler spectrum.

[0012] In some embodiments, before the step of generating a radar echo signal based on the target area, the method further comprises:

[0013] Sending a radio frequency signal to the target area based on the MIMO radar;

[0014] Acquire a radar echo signal of the target area in response to the radio frequency signal.

[0015] In some embodiments, the MIMO radar includes multiple transmitting antennas and multiple receiving antennas, and the frequency modulated continuous wave of the MIMO radar operates in a frequency band ranging from 2.5 GHz to 3.5 GHz, with a signal bandwidth of 1 GHz.

[0016] In some embodiments, generating a two-dimensional image of the target area based on the time domain signal and determining the two-dimensional spatial position of the at least one object from the two-dimensional image includes:

[0017] generating a two-dimensional image of the target area based on backscatter projection according to the time domain signal;

[0018] The two-dimensional spatial position of the at least one object is detected from the two-dimensional image.

[0019] In some embodiments, a constant false alarm rate algorithm is used to detect the two-dimensional spatial position of the at least one object from the two-dimensional image.

[0020] In some embodiments, identifying the behavior of the at least one object based on the image-domain micro-Doppler spectrum includes:

[0021] Extracting preliminary features from the image domain micro-Doppler spectrogram based on a convolution block; the convolution block includes multiple convolution layers, batch normalization, and a ReLU activation function;

[0022] Extracting final features from the preliminary features according to a residual neural network; wherein the residual neural network includes a plurality of stacked residual blocks;

[0023] The behavior of the at least one object is identified from the final features according to the fully connected layer.

[0024] In some embodiments, the residual block includes two cascaded convolutional layers, batch normalization, and a ReLU activation function; and the input features of the next residual block among the multiple residual blocks are features of the residual connection between the output features of the previous residual block and the input features of the previous residual block.

[0025] A second aspect of the embodiments of the present application provides a target behavior recognition device based on space-time-frequency and image-domain micro-Doppler features, the device comprising:

[0026] A time domain signal acquisition module, configured to generate a time domain signal of a target area based on a radar echo signal of the target area; wherein the target area includes at least one object;

[0027] a two-dimensional image generation module, configured to generate a two-dimensional image of the target area according to the time domain signal, and determine the two-dimensional spatial position of the at least one object from the two-dimensional image;

[0028] a three-dimensional image generation module, configured to perform single-point three-dimensional imaging of the at least one object at multiple height points at different height points according to the two-dimensional spatial position, to obtain single-point three-dimensional imaging results at different height points;

[0029] a Doppler map generation module, configured to generate time series data based on the single-point three-dimensional imaging results of any height point of any object within a continuous time, and extract an image-domain micro-Doppler spectrum including amplitude information and phase information of the at least one object from the time series data based on a discrete short-time Fourier transform;

[0030] A target behavior generation module is configured to identify the behavior of the object in the target area according to the image domain micro-Doppler spectrum.

[0031] A third aspect of an embodiment of the present application proposes an electronic device, comprising at least one controller and a memory for communicating with the controller; the memory stores instructions that can be executed by the at least one controller, and the instructions are executed by the at least one controller to enable the at least one controller to perform a target behavior recognition method based on space-time-frequency and image domain micro-Doppler features as described above.

[0032] A fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, it implements the above-mentioned target behavior recognition method based on space-time-frequency and image domain micro-Doppler features.

[0033] The method provided in this embodiment has the following advantages:

[0034] This method first generates a time-domain signal for the target area based on radar echo signals. By employing radar technology, it overcomes the limitations of traditional camera-based behavior recognition in complex lighting conditions. A two-dimensional image of the target area is then generated based on the time-domain signal, and the two-dimensional spatial position of at least one object can be determined from the two-dimensional image. Based on each object's two-dimensional spatial position, single-point three-dimensional imaging is then performed on each object at different heights, resulting in a single-point three-dimensional image. Addressing the signal overlap issue currently encountered in multi-target recognition, this method uses the object's heights to extract single-point image-domain micro-Doppler features. This allows for comprehensive object perception from multiple angles, ensuring rapid and accurate identification of the independent behavior of at least one object in dynamic environments. Finally, based on the single-point three-dimensional imaging results over a continuous time period, an image-domain micro-Doppler spectrum is generated for the at least one object. Based on this image-domain micro-Doppler spectrum, the behavior of the at least one object can be identified. This method improves the accuracy of behavior recognition by combining spatial, temporal, and frequency features through a space-time-frequency collaborative framework, enabling efficient behavior recognition of at least one object in complex scenes.

[0035] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 This is a flowchart of an embodiment of a target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features provided by the present application;

[0038] Figure 2 The micro-Doppler spectra generated by the method provided in the embodiment of the present application for different actions at different height points;

[0039] Figure 3 This is a schematic diagram of the structure of an embodiment of a target behavior recognition model provided by this application;

[0040] Figure 4 Schematic diagram of a confusion matrix embodiment of 7 types of action recognition results provided by this application;

[0041] Figure 5 This is a schematic diagram of an embodiment of the recognition rate of a single target and a dual target in two scenarios provided by this application;

[0042] Figure 6 This is a brief logical block diagram of an embodiment of a target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features provided by this application;

[0043] Figure 7 This is a schematic structural diagram of an embodiment of a target behavior recognition device based on space-time-frequency and image-domain micro-Doppler features provided by the present application;

[0044] Figure 8 It is a structural diagram of an electronic device embodiment provided by this application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0046] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0047] In the description of this application, it should be understood that descriptions involving orientation, such as the orientation or positional relationship indicated by up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0048] like Figure 1 One embodiment of the present application provides a target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features, the method comprising the following steps S110 to S150:

[0049] Step S110 : generating a time domain signal of the target area based on the radar echo signal of the target area.

[0050] In step S110, the target area is the area from which objects need to be extracted and their behaviors identified. The object is the host (e.g., a person, animal, robot, etc.) whose corresponding behavior needs to be identified. Here, the object can be considered a natural person, and the behavior can be considered a human action (e.g., punching or waving). The target area includes at least one object, for example, three natural persons, each performing a different action.

[0051] After acquiring the radar echo signal, the time domain signal of the target area is generated by inverse Fourier transforming the radar echo signal.

[0052] In some embodiments, before the step of generating a radar return signal for the target area, the method further comprises:

[0053] 1) Send radio frequency signals to the target area based on MIMO radar;

[0054] 2) Obtain the radar echo signal of the target area to the radio frequency signal.

[0055] MIMO radar is an ultra-wideband radar technology with many unique advantages, such as high resolution, strong penetration, and good privacy protection. It can provide clear target information in complex environments (such as haze and smoke). It can obtain richer target information and provide multi-dimensional behavioral data.

[0056] The radar echo signal of the MIMO radar is expressed as For the near-field ultra-wideband MIMO radar with linear frequency modulation signal system, the transmission signal of the i-th transmitting antenna can be expressed as ,in is the amplitude-related quantity, is the carrier frequency, is the frequency modulation slope. Let the position be , the position of the i-th transmitting antenna is , the jth receiving antenna position is , two-way distance , the receiving antenna j receives the echo signal from the transmitting antenna i for ,in is the target scattering coefficient, and c is the speed of light.

[0057] In some embodiments, the MIMO radar includes multiple transmitting antenna units and multiple receiving antenna units. The frequency modulated continuous wave of the MIMO radar operates in the frequency band of 2.5 GHz to 3.5 GHz, and the signal bandwidth is 1 GHz, which can enhance the performance of the MIMO radar in complex environments.

[0058] Step S120 , generating a two-dimensional image of the target area according to the time domain signal, and determining the two-dimensional spatial position of at least one object from the two-dimensional image.

[0059] There are many ways to generate a two-dimensional image, such as STFT, CWT, WST, S transform, Wigner distribution, etc. In some embodiments, step S120 includes the following steps S210 to S220:

[0060] Step S210 : generating a two-dimensional image of the target area based on backscatter projection according to the time domain signal.

[0061] Backscatter projection is an imaging technique that reconstructs the spatial structure of backscattered signals from electromagnetic waves or particles interacting with a target. Backscatter projection transforms time-domain signals into a two-dimensional image of the target area. Assuming the radar uses a MIMO system, the following is the detailed implementation of backscatter projection:

[0062] First, the imaging area is divided into grids and the pixel points in the imaging area are calculated. The distance to the radar antenna array and the time delay of transmission are calculated : .

[0063] The radar echo signal received by the antenna array is searched according to the time delay, and the echo signal of each channel is coherently superimposed and calculated.

[0064] The basic formula for backscatter projection imaging is: .in, Indicates location The imaging pixel value at the position, M and N are the number of transmitting antennas and receiving antennas respectively, is the wavelength.

[0065] ‌Produce two-dimensional images of the target area.

[0066] Step S220 : detecting the two-dimensional spatial position of at least one object from the two-dimensional image based on a constant false alarm rate algorithm.

[0067] The Constant False Alarm Rate (CFAR) algorithm is a radar target detection technology that maintains a stable false alarm rate in clutter and interference environments by adaptively adjusting the detection threshold, thereby effectively identifying targets. Targets are detected in imaging results based on the CFAR rate to locate the target in the two-dimensional range-azimuth space.

[0068] Step S130 , performing single-point three-dimensional imaging of multiple height points of at least one target object at different height points according to the two-dimensional spatial position, to obtain single-point three-dimensional imaging results at different height points.

[0069] In step S130 , the position of the object in the range-azimuth two-dimensional space may be extracted from the two-dimensional image, and then single-point three-dimensional imaging of multiple height points may be performed on at least one object at different height points.

[0070] Based on the above embodiment, the detailed operation process is described below:

[0071] Perform single-point three-dimensional imaging on different height points h of the two-dimensional space positions of different detected objects , Represents different objects, , P is the total number of objects, h represents different height points, , H represents the total number of height points selected. Finally, the three-dimensional imaging of the point is performed in chronological order, which is recorded as , where n represents the frame number ( , N is the total number of frames).

[0072] Step S140: Generate time series data based on the single-point three-dimensional imaging results of any height point of any object in a continuous time, and extract an image domain micro-Doppler spectrum including amplitude information and phase information of at least one object from the time series data based on discrete short-time Fourier transform.

[0073] Based on the above embodiment, the single-point 3D imaging result of a certain height point of a certain object constitutes a time series data, which is expressed as .

[0074] Use discrete short-time Fourier transform to analyze this time series data and get the result:

[0075] .

[0076] in, It is about the frame number (discrete time point) n and the discrete frequency number k ( ), It is a discrete window function that intercepts a finite length signal in the time dimension for local spectrum analysis.

[0077] Single-point three-dimensional imaging and short-time Fourier transform are performed on the two-dimensional spatial positions of different objects at different heights in turn, and then the image domain micro-Doppler spectrum combination (i.e., containing amplitude information and phase information) of different objects is obtained.

[0078] Step S150 : Identify the behavior of at least one object according to the micro-Doppler spectrum.

[0079] In some embodiments, step S150 includes the following steps S510 to S530:

[0080] Step S510: extracting preliminary features from the image domain micro-Doppler spectrogram based on a convolution block; wherein the convolution block includes a convolution layer, batch normalization, and a ReLU activation function.

[0081] Step S520: extracting final features from the preliminary features using a residual neural network, wherein the residual neural network includes a plurality of stacked residual blocks.

[0082] Step S530: Identify the behavior of at least one object from the final features according to the fully connected layer.

[0083] This application identifies the behavior of objects in the micro-Doppler spectrum based on a target behavior recognition model. The target behavior recognition model mainly includes:

[0084] (1) Convolutional block; (2) Residual neural network; (3) Fully connected layer;

[0085] In step S510, the convolutional block performs preliminary processing on the micro-Doppler spectrum to extract low-level feature information. Then, step S520 gradually extracts deeper features by stacking multiple residual blocks. Finally, step S530 performs classification through a fully connected layer.

[0086] The target behavior recognition model of the present application can deeply extract the amplitude and phase features in the micro-Doppler image through the combination of convolution and multi-layer residual blocks to improve the classification accuracy.

[0087] The residual block includes two cascaded convolutional layers, batch normalization and ReLU activation function; and the input features of the next residual block in the multiple residual blocks are the features of the residual connection between the output features of the previous residual block and the input features of the previous residual block.

[0088] In each residual block, the input tensor is directly added to the output after the convolution operation through the residual connection, forming a short-circuit path, which promotes the effective learning of deep features.

[0089] The present application provides a target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features, which has at least the following beneficial effects:

[0090] This method first generates a time-domain signal for the target area based on radar echo signals. By employing radar technology, it overcomes the limitations of traditional camera-based behavior recognition in complex lighting conditions. A two-dimensional image of the target area is then generated based on the time-domain signal, and the two-dimensional spatial position of at least one object can be determined from the two-dimensional image. Based on each object's two-dimensional spatial position, single-point three-dimensional imaging is then performed on each object at different heights, resulting in a single-point three-dimensional image. Addressing the signal overlap issue currently encountered in multi-target recognition, this method uses the object's heights to extract single-point image-domain micro-Doppler features. This allows for comprehensive object perception from multiple angles, ensuring rapid and accurate identification of the independent behavior of at least one object in dynamic environments. Finally, based on the single-point three-dimensional imaging results over a continuous time period, an image-domain micro-Doppler spectrum is generated for the at least one object. Based on this image-domain micro-Doppler spectrum, the behavior of the at least one object can be identified. This method improves the accuracy of behavior recognition by combining spatial, temporal, and frequency features through a space-time-frequency collaborative framework, enabling efficient behavior recognition of at least one object in complex scenes.

[0091] like Figures 2 to 6For ease of understanding, taking the example of a human body as the object, a human body action as the behavior, and multiple objects in the area, this application provides a set of embodiments, including the following steps S910 to S940:

[0092] In step S910 , the MIMO radar is initialized, radar echo signals of objects within the target area are acquired, and then an inverse Fourier transform is performed on the radar echo signals to obtain time domain signals.

[0093] Step S920 , performing backscatter projection imaging on the time domain signal to obtain a two-dimensional image, and then performing target detection and target tracking on the two-dimensional image to achieve separation of multiple objects.

[0094] Step S930 , performing single-point 3D imaging on the detected multiple objects at different heights, and then performing short-time Fourier transform on the continuous-time single-point 3D imaging results to construct an image-domain micro-Doppler spectrum of the behavior of the multiple objects.

[0095] In step S940 , the target behavior recognition model is used to extract features from the image domain micro-Doppler spectra of multiple objects, and then the behaviors of the multiple objects are recognized through classification.

[0096] In the target behavior recognition model, high-dimensional feature information in micro-Doppler images is deeply extracted through the combination of convolution and multi-layer residual blocks. In order to effectively extract the amplitude and phase features of micro-Doppler information in the image domain, the target behavior recognition model adopts a residual neural network and combines it with a two-dimensional convolutional neural network to extract features from the micro-Doppler spectrum.

[0097] The target behavior recognition model first performs preliminary processing on the input data through a convolutional block, which consists of a series of convolutional layers, batch normalization, and ReLU activation functions to extract low-level feature information from the input data. Subsequently, the target behavior recognition model progressively extracts deeper features by stacking four residual blocks. Each residual block adopts a BasicBlock structure, consisting of two convolutional layers, batch normalization, and ReLU activation functions. Residual connections directly add the input tensor to the output of the convolution operation, forming a short-circuit path and promoting the effective learning of deep features. Finally, the target behavior recognition model performs classification through a fully connected layer. This layer maps the features extracted by the convolutional and residual modules to the number of categories, completing the task of classifying multiple human behaviors.

[0098] Compared with the prior art, this embodiment has the following beneficial effects:

[0099] 1) This application uses UWB radar's radio frequency signal analysis technology to overcome the limitations of traditional camera-based behavior recognition under complex lighting conditions.

[0100] 2) This application proposes a solution based on space-time-frequency collaborative processing and image-domain micro-Doppler features. This collaborative framework combines spatial, temporal, and frequency features to improve recognition accuracy and system robustness. Compared with traditional methods, this application maintains higher recognition stability and anti-interference capabilities in complex environments (such as multiple people moving and occlusion).

[0101] 3) This application overcomes the signal overlap problem in existing multi-target recognition techniques by employing multiple imaging points for micro-Doppler feature extraction, enabling comprehensive target perception from multiple angles and improving recognition accuracy. Furthermore, this application possesses efficient real-time data processing capabilities, ensuring rapid and accurate identification of the independent behaviors of multiple targets in dynamic environments.

[0102] In order to verify this application, relevant experiments of this embodiment are provided.

[0103] 1) Dataset;

[0104] This embodiment designs a total of 7 types of common human behaviors in life, including punching (A1), kicking (A2), sitting down (A3), bending over (A4), stepping (A5), waving (A6), and extending arms (A7); at the same time, two types of scenes are designed: barrier-free scene (S1) and barrier scene (S2).

[0105] 2) Evaluation indicators;

[0106] This embodiment selects the recognition rate (Acc) of the target as the evaluation indicator of network performance. The recognition rate is defined as the ratio of the number of samples correctly recognized by the model to the total number of samples, that is, , where TP represents the number of samples correctly predicted as positive, FP represents the number of samples incorrectly predicted as positive, TN represents the number of samples correctly predicted as negative, and FN represents the number of samples incorrectly predicted as negative.

[0107] 3) Experimental results;

[0108] As shown in Table 1 below, the collected data is subjected to multi-target feature extraction according to the method used in this embodiment. The number of points at different heights and the use of amplitude and phase information in the frequency domain information and the corresponding overall recognition effect of the fusion of these two types of information are shown in the table. In this embodiment, the point at 1.3m is selected as one point, the points at 0.1m, 0.9m, and 1.7m are selected as a combination of three points, and the points at 0.1m, 0.5m, 0.9m, 1.3m, 1.7m, and 2.1m are selected as a combination of six points. As can be seen from the table, the scheme of extracting micro-Doppler features from multiple height points and fusing amplitude and phase information adopted in this embodiment effectively improves the recognition rate of human behavior recognition.

[0109] Table 1

[0110]

[0111] Figure 4 This is the confusion matrix of the 7 types of behavior recognition results of this embodiment. The experimental results prove that after completion, this application shows good recognition ability for the target recognition rate, overcoming the challenge that the traditional micro-Doppler feature extraction method cannot achieve multi-target separation and recognition. The human behavior feature characterization method proposed in this embodiment can effectively represent subtle changes in human behavior.

[0112] Figure 5 The following are the recognition rates for single and dual targets in two scenarios for this embodiment. The test results show that the recognition rate for multi-person data is slightly lower than that for single-person data. This is because in multi-person scenarios, targets may affect each other, causing changes in Doppler characteristics during imaging, thereby reducing the recognition rate. Compared to unobstructed scenarios, the recognition rate in obstructed scenarios decreases because the interference from obstacles affects the propagation of electromagnetic waves, resulting in a lower recognition rate. Despite the decrease in recognition rate across different scenarios, the recognition rate remains above 85%.

[0113] like Figure 7 One embodiment of the present application provides a target behavior recognition device based on space-time-frequency and image-domain micro-Doppler features, the device comprising:

[0114] The time domain signal acquisition module 1100 is used to generate a time domain signal of the target area based on the radar echo signal of the target area; wherein the target area includes at least one object;

[0115] The two-dimensional image generation module 1200 is used to generate a two-dimensional image of the target area according to the time domain signal, and determine the two-dimensional spatial position of at least one object from the two-dimensional image;

[0116] The 3D image generation module 1300 is configured to perform single-point 3D imaging of multiple height points of at least one object at different height points according to the 2D spatial position, thereby obtaining single-point 3D imaging results at different height points.

[0117] The Doppler map generation module 1400 is used to generate time series data based on the single-point three-dimensional imaging results of any height point of any object in a continuous time, and extract an image domain micro-Doppler spectrum including amplitude information and phase information of at least one object from the time series data based on discrete short-time Fourier transform;

[0118] The target behavior generating module 1500 is configured to identify the behavior of at least one object based on the micro-Doppler spectrum.

[0119] It should be noted that the present embodiment of the target behavior recognition device based on space-time-frequency and image domain micro-Doppler features and the above-mentioned embodiment of the target behavior recognition method based on space-time-frequency and image domain micro-Doppler features are based on the same inventive concept. Therefore, the relevant contents of the above-mentioned embodiment of the target behavior recognition method based on space-time-frequency and image domain micro-Doppler features are also applicable to the present embodiment of the target behavior recognition device based on space-time-frequency and image domain micro-Doppler features, and will not be repeated here.

[0120] Reference Figure 8 , an embodiment of the present application further provides an electronic device, the electronic device comprising:

[0121] at least one memory;

[0122] at least one processor;

[0123] at least one program;

[0124] The programs are stored in the memory, and the processor executes at least one program to implement the target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features described above.

[0125] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0126] The electronic device according to the embodiment of the present application is described in detail below.

[0127] The processor 1600 may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0128] Memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 1700 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the program code is stored in memory 1700 and is called by processor 1600 to execute the target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features in the embodiments of this application.

[0129] Input / output interface 1800, used for information input and output;

[0130] Communication interface 1900, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0131] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0132] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0133] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, which are used to enable a computer to execute the above-mentioned target behavior recognition method based on space-time-frequency and image domain micro-Doppler features.

[0134] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0136] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0137] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0138] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0139] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0140] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features, characterized by: The method comprises: generating a time domain signal of the target area based on a radar echo signal of the target area; wherein the target area includes at least one object; generating a two-dimensional image of the target area according to the time domain signal, and determining a two-dimensional spatial position of the at least one object from the two-dimensional image; performing single-point three-dimensional imaging of the at least one object at multiple height points at different height points according to the two-dimensional spatial position to obtain single-point three-dimensional imaging results at different height points; generating time series data based on the single-point three-dimensional imaging results of any height point of any object in a continuous time, and extracting an image-domain micro-Doppler spectrum including amplitude information and phase information of the at least one object from the time series data based on a discrete short-time Fourier transform; The behavior of the object in the target area is identified according to the image-domain micro-Doppler spectrum.

2. The target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to claim 1 is characterized in that: Before the step of generating a radar echo signal based on the target area, the method further includes: Sending a radio frequency signal to the target area based on the MIMO radar; Acquire a radar echo signal of the target area in response to the radio frequency signal.

3. The target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to claim 2 is characterized in that: The MIMO radar includes multiple transmitting antennas and multiple receiving antennas, and the frequency modulated continuous wave of the MIMO radar operates in the frequency band of 2.5 GHz to 3.5 GHz, with a signal bandwidth of 1 GHz.

4. The target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to claim 1 is characterized in that: Generating a two-dimensional image of the target area according to the time domain signal, and determining the two-dimensional spatial position of the at least one object from the two-dimensional image includes: generating a two-dimensional image of the target area based on backscatter projection according to the time domain signal; The two-dimensional spatial position of the at least one object is detected from the two-dimensional image.

5. The target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to claim 4 is characterized in that: A constant false alarm rate algorithm is used to detect the two-dimensional spatial position of the at least one object from the two-dimensional image.

6. The target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to claim 1 is characterized in that: The identifying the behavior of the at least one object according to the image-domain micro-Doppler spectrum includes: Extracting preliminary features from the image domain micro-Doppler spectrogram based on a convolution block; the convolution block includes multiple convolution layers, batch normalization, and a ReLU activation function; Extracting final features from the preliminary features according to a residual neural network; wherein the residual neural network includes a plurality of stacked residual blocks; The behavior of the at least one object is identified from the final features according to the fully connected layer.

7. The target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to claim 6 is characterized in that: The residual block includes two cascaded convolutional layers, batch normalization and a ReLU activation function; and the input features of the next residual block among the multiple residual blocks are the features of the residual connection between the output features of the previous residual block and the input features of the previous residual block.

8. A target behavior recognition device based on space-time-frequency and image-domain micro-Doppler features, characterized by: The device comprises: A time domain signal acquisition module, configured to generate a time domain signal of a target area based on a radar echo signal of the target area; wherein the target area includes at least one object; a two-dimensional image generation module, configured to generate a two-dimensional image of the target area according to the time domain signal, and determine the two-dimensional spatial position of the at least one object from the two-dimensional image; a three-dimensional image generation module, configured to perform single-point three-dimensional imaging of the at least one object at multiple height points at different height points according to the two-dimensional spatial position, to obtain single-point three-dimensional imaging results at different height points; a Doppler map generation module, configured to generate time series data based on the single-point three-dimensional imaging results of any height point of any object within a continuous time, and extract an image-domain micro-Doppler spectrum including amplitude information and phase information of the at least one object from the time series data based on a discrete short-time Fourier transform; A target behavior generation module is configured to identify the behavior of the object in the target area according to the image domain micro-Doppler spectrum.

9. An electronic device, characterized in that: The invention comprises at least one controller and a memory for communicating with the controller; the memory stores instructions that can be executed by the at least one controller, and the instructions are executed by the at least one controller to enable the at least one controller to perform the target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the target behavior recognition method based on space-time-frequency and image-domain micro-Doppler features according to any one of claims 1 to 7.

Images