Open-set gait recognition method under multi-vision and multi-wearing based on millimeter wave radar

Abstract

The application discloses a kind of open set gait recognition methods under multi-vision multi-wearing based on millimeter wave radar, including obtaining human gait information collection radar echo signal, and the echo signal obtained is preprocessed, while constructing spectrum set;Gait recognition network model based on millimeter wave radar is constructed;Using the constructed spectrum set, by joint calculation of several loss functions, the gait recognition network model based on millimeter wave radar constructed is trained and optimized;Set open set model sample library, using the gait recognition network model based on millimeter wave radar after training and optimization, based on echo signal, complete gait recognition processing;The accuracy of the method of the application is improved, the practical applicability is enhanced, and the robustness is enhanced.

Claims

1. An open-set gait recognition method based on millimeter-wave radar under multi-vision and multi-wearable conditions, comprising the following steps: S1. Acquire human gait information by collecting radar echo signals, preprocess the acquired echo signals, and construct a spectrum atlas. S2. Construct a gait recognition network model based on millimeter-scale radar; the constructed gait recognition network model based on millimeter-scale radar includes a multi-scale temporal extraction structure layer; through the multi-scale temporal extraction structure, the input is processed to obtain feature spectra in three different temporal dimensions; the three feature spectra are short-time feature spectra. Normal characteristic spectrum Long-term characteristic spectrum Short-time characteristic spectrum The dimension is ; Characteristic spectrum under normal conditions The dimension is Long-term characteristic spectrum The dimension is ;in, This indicates the number of batches in each training iteration of the model; Indicates the number of channels in the image; The dimension representing the time axis of the millimeter-wave spectrum; The dimension representing the frequency axis of the millimeter-wave spectrum; S3. Using the spectrum atlas constructed in step S1, train and optimize the gait recognition network model based on millimeter radar constructed in step S2 by jointly calculating several loss functions; Several loss functions are jointly calculated, specifically including: The jointly calculated loss functions include the gait loss function, the ternary loss function, and the center loss function; (1) Gait loss function: The gait loss function is related to the last layer of the gait recognition network model; Set the hidden layer size to The fully connected layer, in which, Indicates the number of predicted images; The gait loss function is calculated using the following formula: in, Represents the gait loss function; Indicates the actual gait label; Indicates the gait label predicted by the model; The model predicts as The probability of; Indicates a constant; (2) Ternary loss function: The purpose of calculating the ternary loss function is to establish a triplet (anchor, positive, negative) consisting of three samples. By training the ternary loss function in the triplet, the goal is to achieve the following: the feature vectors of the same sample are closest to each other, and the feature vectors of different samples are farthest from each other. The ternary loss function is calculated using the following formula: in, Represents the ternary loss function; Indicates Euclidean distance; The feature vector representing the benchmark sample; The feature vector representing a positive sample; The feature vector representing a negative sample; Indicates boundary values; The training objective of the model is to minimize the Euclidean distance. At the same time, maximize the Euclidean distance This results in the feature vectors of the same sample being closest to each other, and the feature vectors of different samples being farthest apart. The samples are sampled to train the ternary loss function, ensuring that each batch of the dataset contains an integer multiple of the triples; Through boundary values Optimize the ternary loss function; (3) Central loss function: By learning the deep feature centers of different categories, the distance between the deep features and the corresponding category centers is penalized; The central loss function is calculated using the following formula: in, Represents the central loss function; Indicates batch size; Indicates the first The central characteristics of each category; Indicates the first The image's tag; The center loss function is used to represent the differences between classes, thereby increasing the compactness within classes; S4. Set up an open set model sample library, and use the gait recognition network model based on millimeter radar trained and optimized in step S3 to complete the gait recognition process based on the echo signal.

2. The open-set gait recognition method based on millimeter-wave radar under multi-vision and multi-wearable conditions according to claim 1, characterized in that... Step S1, which involves acquiring the echo signal, preprocessing the acquired echo signal, and constructing a spectrum atlas, specifically includes: The receiver and transmitter perform frequency mixing processing to obtain the regulated human gait information radar echo signal; For the acquired radar echo signal, a fast Fourier transform is performed along the range dimension to obtain a time-range spectrum. Static clutter removal processing is performed on the acquired time-range spectrum to eliminate clutter; Select a unit within the personnel location range on the time range graph and perform a short-time Fourier transform; the short-time Fourier transform is expressed by the following formula: in, This represents the short-time Fourier transform of the echo signal in the time domain t and the frequency domain ω; Indicates the time dimension; Indicates the frequency dimension; This represents the received echo spectrum; Indicates the window function; Constructing a spectrogram set: The constructed spectrogram set contains data from several subjects walking along several different viewpoints under set clothing conditions; the spectrogram sets obtained by performing short-time Fourier transform on the collected echo signals are compiled into a spectrogram set, and each spectrogram in the spectrogram set shows the distribution of the signal in the frequency domain.

3. The open-set gait recognition method based on millimeter-wave radar under multi-vision and multi-wearable conditions according to claim 2, characterized in that... Step S2, which involves constructing a gait recognition network model based on millimeter radar, specifically includes: The gait recognition network model based on millimeter radar includes an input layer, a multi-scale temporal extraction structure layer, a ResNet18 layer, a splicing layer, a global average pooling layer, a batch normalization layer, and a classifier layer. 1) Input layer: Millimeter-wave spectra are selected as the input to the network model and fed into the input layer of the model. Select indicates input. ; 2) ResNet 18 layers: The ResNet18 layer processes the feature spectra from the multi-scale temporal extraction structure layer, which has three different temporal dimensions, to extract gait features. Using the ResNet18 network, short-time feature spectral maps are targeted. Gait feature extraction is performed to obtain the processed feature spectrum. ;in, This indicates the number of channels in the output spectrum after short-time feature map extraction; This indicates the dimension of the time axis of the output spectrum after short-time feature extraction; This indicates the dimension of the frequency axis of the output spectrum after short-time feature spectrum extraction; Using the ResNet18 network, the feature spectrum during normal time is analyzed. Gait feature extraction is performed to obtain the processed feature spectrum. ;in, This indicates the number of channels in the output spectrum after feature spectrum extraction under normal conditions; This indicates the dimension of the time axis of the output spectrum after feature spectrum extraction under normal conditions; This indicates the dimension of the frequency axis of the output spectrum after feature spectrum extraction under normal conditions; Using the ResNet18 network, long-term feature spectral maps are targeted. Gait feature extraction is performed to obtain the processed feature spectrum. ;in, This indicates the number of channels in the output spectrum after long-term feature spectrum extraction; This indicates the dimension of the time axis of the output spectrum after long-term feature spectrum extraction; This indicates the dimension of the frequency axis of the output spectrum after long-term feature spectrum extraction; 3) Splicing layer: The concatenation layer is the feature spectrum obtained after processing ResNet18 layers. , , The feature spectrum is obtained by concatenating the data over time. ;in, This indicates the number of channels in the spliced ​​feature spectrum; This indicates the dimension of the frequency axis of the spliced ​​feature spectrum; The following formula represents the characteristic spectrum after splicing. : in, This indicates a splicing operation along the time dimension; 4) Global average pooling layer: Global average pooling layers are used for the feature spectra obtained after splicing. Perform global average pooling to obtain the feature spectrum after global average pooling. ;in, This represents the number of channels in the feature map after global average pooling. 5) Batch standardization layer: The batch normalization layer performs batch normalization on the feature spectrum obtained by global average pooling to obtain a normalized feature spectrum. 6) Classifier layer: The classifier layer processes the batch-standardized data, feeding the batch-standardized data into the classifier for discrimination, and obtaining output features. The input layer, multi-scale temporal extraction structure layer, ResNet18 layer, splicing layer, global average pooling layer, batch normalization layer, classifier layer, are connected in series.

4. The open-set gait recognition method based on millimeter-wave radar under multi-vision and multi-wearable conditions according to claim 3, characterized in that... Step S3, which uses the spectrum atlas constructed in step S1, trains and optimizes the gait recognition network model based on millimeter radar constructed in step S2 by jointly calculating several loss functions. Specifically, this includes: (3-1) Model training and optimization: In the experiment, the input spectrogram was scaled up and the image size was adjusted proportionally while keeping the relative position of the image content unchanged, so as to adapt to the input requirements of the model. Meanwhile, the Adam optimizer is used to optimize the ternary loss function in the jointly calculated loss function; Through training, gait features from several perspectives are obtained and stored in the sample database; (3-2) Loss function calculation: By jointly calculating several loss functions, the parameters of the gait recognition network model based on millimeter radar are updated to learn accurate micro-motion features. The jointly calculated loss functions include the gait loss function, the ternary loss function, and the center loss function; The following formulas represent several loss functions used for joint calculation: in, This represents several loss functions used for joint computation; Adjustment parameters representing center loss; Represents the gait loss function; Represents the ternary loss function; This represents the central loss function.

Images