Gait identity recognition method based on generative self-supervised visual pre-training model BEiT and DAS technology

Abstract

The invention discloses a gait identity recognition method based on a generative self-supervised visual pre-training model BEiT and DAS technology. The method comprises the following steps: firstly, carrying out scene construction and data preparation, collecting label-free data and label data, extracting data signals for signal preprocessing, and constructing a time frequency characteristic data set; constructing a pre-training image reconstruction network of a self-supervised visual pre-training model BEiT, performing feature extraction on an input image, constructing a deep learning network based on the self-supervised visual pre-training model BEiT as a feature extractor, and training the deep learning network by using unlabeled spatio-temporal data in an experimental scene to obtain weight parameters; and constructing a downstream classification task network, performing classification learning of downstream tasks by using a small amount of data with labels to obtain a new weight, and completing identification and classification of non-label data by using the obtained new weight. According to the method, the demand quantity of the tagged data is reduced, and the accuracy is ensured.

Description

Technical Field

[0001] This invention belongs to the field of distributed fiber optic acoustic vibration sensing technology, and relates to a gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology. Background Technology

[0002] Distributed Acoustic Sensing (DAS), using acoustic sensors as its core technology, is increasingly becoming a focus of sensing research due to its wide coverage and high information acquisition efficiency. It is primarily applied in the energy sector, such as natural gas and oil extraction and development, as well as in important fields like military and defense. However, the massive amounts of data generated by DAS technology are difficult to process and analyze using traditional methods. In recent years, big data and deep learning methods have been increasingly applied to the analysis and processing of DAS data, making data analysis and processing more accurate, faster, and more comprehensive.

[0003] Gait measurement using DAS is more discreet and less privacy-involving than using cameras, and it has many long-distance, large-area measurement capabilities that conventional gait measurement lacks (Reference 1: Li Yikuan. A method for gait feature recognition in early Parkinson's disease based on multi-feature fusion [J]. Automation and Information Engineering, 2024, 45(01): 55-60). Currently, there is relatively little research in this area. Recently, the National University of Singapore implemented footprint recognition on the DAS dataset and evaluated the effectiveness of DAS as a footprint detection method (Reference 2: Zhou, Yuhan, et al. Large scale indoor occupant tracking using distributed acoustic sensing and machine learning). Building and Environment 247(2024): 111005. To this end, a footprint recognition method based on neural networks is proposed. In addition, a comparative experiment was conducted using a seismic detector as a benchmark. This comparison includes footstep recognition and further data analysis, including sensing distance estimation, walking pattern estimation, and speed calculation. The results demonstrate the effectiveness of DAS in pedestrian detection, and it provides more detailed information compared to detectors. However, the entire measurement is based on single-person gait information, which is clearly impractical for real-world scenarios. Researchers at Shantou University have developed a dual-YOLO method based on stride length and stride frequency to determine pedestrian IDs (Reference 3: Shi, Yi, et al. "Footsteps detection and identification based on distributed optical fibersensor and double-YOLO model." Optics Express 31.25(2023):41391-41405). Preliminary field experiments show that this method can detect, locate, and identify the footprints of three people, with an accuracy of approximately 86.0%, which is 12.6% higher than the single-YOLO method. This gait detection and identification method can promote the development of gait-based clinical diagnosis or person recognition applications. However, it is clear that this recognition rate still needs to be improved in practical applications. Currently, there are patents for gait identification based on DAS technology. One invention patent, "A Human Gait Recognition Method Based on a Self-Supervised Feature Mask Autoencoder in a Distributed Fiber Optic Sensing System" (Publication No. CN119863657A), uses a self-supervised feature mask autoencoder, combined with a small amount of labeled data and a large amount of unlabeled data, to identify various gait identities. However, the overall recognition accuracy is not ideal. Therefore, gait identification based on DAS technology can achieve large-scale and long-distance recognition.

[0004] Currently, most DAS data processing employs supervised learning-type deep learning methods. However, supervised learning requires all data to be labeled before training and optimization can be performed. In practical DAS applications, acquiring large amounts of labeled DAS data is costly or extremely difficult. The reasons include: (1) collecting DAS data from various sources or fields can be time-consuming and costly; (2) some DAS data may be restricted by privacy regulations, making them difficult to obtain; (3) some DAS detection tasks may require specialized knowledge or skills to accurately label or annotate the data; and (4) most existing DAS databases are unlabeled.

[0005] Therefore, how to use AI models to achieve efficient and accurate identification of DAS gait signals in the face of a large amount of unlabeled data collected by DAS devices in practice is an urgent problem to be solved. Summary of the Invention

[0006] To address the challenge of obtaining sufficient labeled DAS data for identification tasks in distributed fiber optic sensing systems, this invention provides a gait identification method based on the generative self-supervised visual pre-training model BEiT and DAS technology. The pre-training network requires only a sufficient amount of unlabeled data, while the gait recognition network requires only a small amount of labeled data, thus reducing the amount of labeled data needed while ensuring accuracy.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] A gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology includes the following steps:

[0009] Step 1, Scene Setup: Set up a distributed fiber optic vibration sensing system on the back of the carpet. Arrange optical fibers at certain intervals on the back of the carpet of a set size and connect the optical fibers to the distributed fiber optic sensing system.

[0010] Step 2, Data Preparation: Different participants walked randomly on the carpet, and a distributed fiber optic vibration sensing system was set up on the carpet to collect gait signals of the participants walking freely.

[0011] Step 3, Signal Preprocessing: Collect unlabeled and labeled data, extract data signals for signal preprocessing, and construct a time-frequency feature dataset;

[0012] Step 4: Construct a pre-trained image reconstruction network for the self-supervised visual pre-trained model BEiT. The pre-trained network includes a word segmenter T1, a masker M, an encoder E1, and a decoder D. It extracts features from the input image and uses the above deep learning network to construct a feature extractor based on the self-supervised visual pre-trained model BEiT. It is then trained using unlabeled spatiotemporal data from the experimental scenario to obtain the weight parameters.

[0013] Step 5: Construct a downstream classification task network. Use a small amount of labeled data to learn the classification of downstream tasks and obtain new weights. Use the obtained new weights to complete the identification and classification of unlabeled data.

[0014] Furthermore, step 3 includes the following sub-steps:

[0015] 3.1 Based on the data signal transmitted through the optical fiber, wavelet filtering is used to reduce noise in the signal and minimize the impact of low frequencies on the intrusion signal;

[0016] 3.2 The distributed fiber optic vibration sensing system returns the original signal trajectory distributed in space at each moment, continuously accumulates the original signal on the time axis, performs Fourier transform on the signal, and constructs a time and frequency signal matrix;

[0017] 3.3. Through data processing methods, the obtained spatiotemporal signal matrix is ​​transformed into a 224×224 spatiotemporal graph, resulting in spatiotemporal graphs of four types of gait events in each scenario, with 2000 samples of each type of event in each scenario.

[0018] 3.4 For data in laboratory scenarios, half of the data is used as pre-training data, and the other half is used as the validation set after downstream training.

[0019] Furthermore, step 4 includes the following steps:

[0020] 4.1 Image Block Processing: The spatiotemporal image of the DAS data is divided into fixed-size image blocks and flattened to obtain a block sequence. in Let be the flattened vector of the i-th block of the labeled sequence;

[0021] 4.2 Random Masking by the Masker: The masker M randomly and uniformly selects a portion of the image patch as the center point according to the corresponding label group. Based on each center patch, it expands into a continuous square region of fixed size (e.g., 3×3). This region is then masked. The mask is represented by a learnable label sequence. The masking ratio needs to be moderate to reduce information redundancy between image patches, which is more conducive to the model network learning the global features of the image. The masked sequence is denoted as... Simultaneously, a pre-trained image tagger, i.e., a word segmenter T1, is introduced, typically a Discrete Variational Autoencoder (dVAE), to segment image patches. Quantized into discrete visual markers y i ∈{1, ...,K}, where K is the codebook size, and the tokenization process is as follows:

[0022]

[0023] in It is a discrete codebook.

[0024] 4.3 Linear Projection and Position Encoding: The masked image patches are input into the linear projection layer. A learnable linear transformation maps the image patches to the latent space, and position encoding is added to preserve spatial information. A label corresponding to each image patch is generated, and an ordered label sequence with embedded position encoding is output, resulting in a label sequence with position encoding. Where E is the learnable projection matrix, used to transform local features into high-level semantics. This is a location embedding vector used to encode spatial location information. The complete input sequence is:

[0025] 4.4 Encoder Encoding: The encoder E1 part consists of L stacked coding blocks. Each coding block first enters the normalization layer to output a normalized label sequence. The normalized label sequence then enters the multi-head self-attention layer. In the self-attention layer, the input sequence X is first subjected to three linear transformations to obtain the sequence of query Q, key K, and value V, which can be obtained through matrix multiplication and projection matrix W. Q W K W V To obtain:

[0026]

[0027] Among them W Q W K W V It is a weight matrix obtained through training. The obtained sequence of query Q, key K, and value V is divided into h heads, resulting in head q. i ,k i ,v i This is a subset of the original sequence. An attention score is calculated for each head i using a dot product attention mechanism:

[0028]

[0029] in It is the scaling factor, d kThis represents the dimensions of query Q and key K in each head. The Softmax function converts the attention scores into weights. Concatenating the attention outputs of each head yields the final output of the multi-head self-attention mechanism.

[0030] MultiHead(Q,K,V)=Concat(Attention(q1,k1,v1),Attention(q2,k2,v2),...,Attention(q h ,k h ,v h Each coding block contains a multi-head self-attention layer and a feedforward network (FFN). The operation of the l-th layer is as follows:

[0031] Z′ l =MultiHead(LayerNorm(Z) l-1 ))+Z l-1 ;

[0032] Z l =FFN(LayerNorm(Z′) l ))+Z′ l ;

[0033] Z l-1 Given the input sequence, LayerNorm is a layer normalization operation. Multi-head self-attention maintains the same output dimension. FFN is defined as:

[0034] FFN(U)=GELU(UW1+b1)W2+b2;

[0035] Where W and b are the trainable weight matrix and bias matrix, respectively. The output after multi-head self-attention concatenation undergoes a linear transformation through the trainable weight matrix W, and is then residually concatenated with the input sequence to finally output Z. l It includes global features ([CLS] position) and local features (positions of each block);

[0036] 4.5 Decoder Decoding: After E1 encoding by the encoder, the output feature matrix Z is completed. l Let the set of mask positions be... From Z l Extract mask block features:

[0037]

[0038] Then, the visual marker probability distribution of the mask block is predicted using decoder D:

[0039]

[0040] Where W is the learnable weight matrix, K is the size of the visual tag codebook, and yi For real visual marking;

[0041] The pre-training task uses masked image modeling, with the objective being to model based on masked block sequences. Reconstruct its visual token y i The loss function used is the cross-entropy function:

[0042]

[0043] Where y i It is the true visual marker of the i-th mask block. It is the final encoded feature of the i-th mask block. The optimization objective is to minimize the predicted negative log-likelihood of all mask blocks, that is, to maximize the labeled predicted likelihood of the mask block.

[0044] Step 5 includes the following sub-steps:

[0045] 5.1 The initial weight parameters of the encoder E2 constructed by the downstream task are the same as the weight parameters of the encoder E1 of the pre-training task after training.

[0046] 5.2 In the position encoding step, the data tag value cls is added according to the tag data, the 0 position information is embedded, and it is concatenated with the tag sequence output by linear projection and input into the encoder output with the tag data;

[0047] 5.3. Use the tag value represented by cls as the final output of encoder E2;

[0048] 5.4 Calculate the cross-entropy loss function between the encoder output and the original data label values:

[0049]

[0050] Where y c For one-hot encoding of the real label, p c To predict probabilities.

[0051] The beneficial effects of this invention are mainly reflected in the following: The use of machine learning models for event pattern recognition in DAS systems has wide applications in its field. These machine learning models typically require a large amount of labeled data; however, obtaining a large amount of labeled data in practice is costly and potentially very difficult. Self-supervised learning methods are a relatively effective solution. This invention targets DAS systems and, based on a generative self-supervised visual pre-training model BEiT, sets up a pre-trained network for reconstructing visual labels corresponding to unlabeled DAS data and a pattern recognition visual ViT network based on a small amount of labeled data. This effectively reduces the need for labeled data in the DAS system pattern recognition machine learning model, and effectively combines unlabeled data to achieve event pattern recognition without sacrificing accuracy. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0053] Figure 2 This is a block diagram of a DAS data acquisition system;

[0054] Figure 3 This is a schematic diagram of experimental data acquisition.

[0055] Figure 4 This is a spatiotemporal diagram of the DAS gait signal of A after preprocessing;

[0056] Figure 5 This is a spatiotemporal diagram of the DAS gait signal of B after preprocessing;

[0057] Figure 6 This is a spatiotemporal diagram of the DAS gait signal of C after preprocessing;

[0058] Figure 7 This is a spatiotemporal diagram of the DAS gait signal of D after preprocessing;

[0059] Figure 8 This is a diagram of the pre-trained network structure.

[0060] Figure 9 This is a diagram of the downstream task network structure.

[0061] Figure 10 Accuracy plots for downstream tasks with different numbers of labels;

[0062] Figure 11 The confusion matrix diagram for pattern recognition prediction in a self-supervised network model. Detailed Implementation

[0063] The present invention will now be further described with reference to the accompanying drawings.

[0064] Reference Figures 1 to 11This paper presents a gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology. The method performs pattern recognition on four different gait patterns of four participants, illustrating the application of a self-supervised learning-based distributed fiber optic sensing system gait pattern recognition method in DAS system pattern recognition scenarios. This example uses unlabeled data containing the four types of events and labeled data with corresponding tags to classify the four types of gait signals acquired by DAS through a self-supervised learning model.

[0065] The gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology includes the following steps:

[0066] Using laboratory carpets as a specific example, the flowchart of the gait recognition method based on distributed fiber optic sensing is as follows: Figure 1 As shown

[0067] Step 1: Scenario setup, prepare the DAS system, collect unlabeled and labeled data, extract data signals and perform signal preprocessing.

[0068] Building such Figure 2 The experimental setup for the DAS data acquisition system shown integrates DAS components such as a narrow-linewidth laser, an optical amplifier (EDFA), an optical coupler, a local oscillator, an acousto-optic modulator, an optical circulator, an optical filter, a photodetector, and a high-speed data acquisition card into a DFS-2000 series DAS demodulation instrument. This instrument, model DFS-2005, has a detection distance of 5 km, a spatial resolution of 5 cm, and a DAS signal sampling rate of 20 kS / s. The instrument is connected to a 5 km long G652 rubber-coated single-mode optical fiber containing 959 fiber optic points.

[0069] The laser output from the laser is modulated and amplified to form a probe pulse, which enters the sensing fiber to scan the fiber optic link. Based on changes in the scattered signal at different times, vibration events can be located. Backscattered light in the sensing fiber is collected by a circulator and interferes with the local oscillator light to obtain the phase information of the scattered light. After passing through a photodetector and a digital-to-analog converter chip, the signal is processed by an FPGA, including phase demodulation and fading suppression, ultimately obtaining vibration information at different locations in the fiber. The demodulated vibration signal is transmitted to an industrial control computer via gigabit Ethernet for backend processing such as signal characteristic analysis and event identification.

[0070] The optical fibers are laid parallel to each other on the carpet at 5cm intervals. One end of the fiber is connected to the distributed optical system (DAS), and the other end is coiled. The fiber arrangement method on the carpet is as follows: Figure 3 As shown.

[0071] Step 2: Data preparation, collecting free walking signals from 4 experimenters:

[0072] Data collection without labels: After starting the machine and setting the parameters, data was collected at seven fiber optic points: 35, 36, 37, 38, 39, 40, and 41. The collection time was 15,000 seconds, which included four types of events, and a total of 15,000 × 20,000 × 7 data values ​​were collected as the unlabeled dataset.

[0073] Collect labeled data: After starting the machine and setting the parameters, data was collected at seven fiber optic points: 35, 36, 37, 38, 39, 40, and 41. The collection time was 1000 seconds. The data included four types of events, and a total of 1000×20000×7 data values ​​were collected as labeled dataset.

[0074] Step 3, Signal Preprocessing: Collect unlabeled and labeled data, extract data signals for signal preprocessing, and construct a time-frequency feature dataset;

[0075] Step 3 includes the following sub-steps:

[0076] 3.1 Based on the data signal transmitted through the optical fiber, wavelet filtering is used to reduce noise in the signal and minimize the impact of low frequencies on the intrusion signal;

[0077] 3.2 The distributed fiber optic vibration sensing system returns the original signal trajectory distributed in space at each moment, continuously accumulates the original signal on the time axis, performs Fourier transform on the signal, and constructs a time and frequency signal matrix;

[0078] 3.3. Through data processing methods, the obtained spatiotemporal signal matrix is ​​transformed into a 224×224 spatiotemporal graph, resulting in spatiotemporal graphs of four types of gait events in each scenario, with 2000 samples of each type of event in each scenario.

[0079] 3.4 For data in laboratory scenarios, half of the data is used as pre-training data, and the other half is used as the validation set after downstream training.

[0080] At fiber optic points 35, 36, 37, 38, 39, 40, and 41 where the event occurred, corresponding to row indices of the spatiotemporal two-dimensional signal matrix, data signals were extracted. Each signal had a duration of 5 seconds. Wavelet transform was used to denoise the data, eliminating noise in the signal in the frequency domain, achieving a noise reduction effect. The principle formula is as follows:

[0081]

[0082] Where a>0, it is called the scaling factor, which plays a role in the scaling of the basic wavelet. The function is scaled, where b is the scaling factor, which is the distance of the displacement.

[0083] The steps of Continuous Wavelet Transform (CWT) are as follows: First, select a wavelet basis function, fix a scale factor, and compare it with the initial segment of the signal; calculate the wavelet coefficients (which reflect the similarity between the wavelet at the current scale and the corresponding signal segment) using the CWT calculation formula; change the translation factor to shift the wavelet along the time axis, and repeat the above two steps to complete one analysis; increase the scale factor and repeat the above three steps for a second analysis; repeat the above four steps three times to obtain a clean signal.

[0084] After decomposing the wavelet into four levels, a thresholding process is performed. The formula for selecting the threshold is:

[0085]

[0086] Where f(k) min The absolute value of each element in the signal is taken, and then sorted from largest to smallest. The new signal sequence is obtained by squaring each sequence.

[0087] The thresholded wavelet signal is subjected to an inverse continuous wavelet transform (CWT) to obtain a clean signal. A 256-point fast Fourier transform is performed on the denoised event signal to obtain its frequency domain amplitude. Plotting time on the horizontal axis and frequency on the vertical axis, with color depth representing the frequency domain amplitude, yields the corresponding spatiotemporal graph of the event signal, as shown below. Figure 4-7 As shown.

[0088] The unlabeled dataset contains 3,000 spatiotemporal graphs, while the labeled dataset contains 200 spatiotemporal graphs, with 50 spatiotemporal graphs for each event category.

[0089] Step 4: Construct a pre-trained image reconstruction network based on the self-supervised visual pre-trained model BEiT: Construct a pre-trained image reconstruction network based on the self-supervised visual pre-trained model BEiT, including a word segmenter T1, a masker M, an encoder E1, and a decoder D. Extract features from the input image, and use the above deep learning network to construct a feature extractor based on the self-supervised visual pre-trained model BEiT. Train it using unlabeled spatiotemporal data from the experimental scenario to obtain the weight parameters.

[0090] Step 4 includes the following steps:

[0091] 4.1 Image Block Processing: The spatiotemporal image of the DAS data is divided into fixed-size image blocks and flattened to obtain a block sequence. in Let be the flattened vector of the i-th block of the labeled sequence;

[0092] 4.2 Random Masking by the Masker: The masker M randomly and uniformly selects a portion of the image patch as the center point according to the corresponding label group. Based on each center patch, it expands into a continuous square region of fixed size (e.g., 3×3). This region is then masked. The mask is represented by a learnable label sequence. The masking ratio needs to be moderate to reduce information redundancy between image patches, which is more conducive to the model network learning the global features of the image. The masked sequence is denoted as... Simultaneously, a pre-trained image tagger, i.e., a word segmenter T1, is introduced, typically a Discrete Variational Autoencoder (dVAE), to segment image patches. Quantized into discrete visual markers y i ∈{1, ...,K}, where K is the codebook size, and the tokenization process is as follows:

[0093]

[0094] in It is a discrete codebook.

[0095] 4.3 Linear Projection and Position Encoding: The masked image patches are input into the linear projection layer. A learnable linear transformation maps the image patches to the latent space, and position encoding is added to preserve spatial information. A label corresponding to each image patch is generated, and an ordered label sequence with embedded position encoding is output, resulting in a label sequence with position encoding. Where E is the learnable projection matrix, used to transform local features into high-level semantics. This is a location embedding vector used to encode spatial location information. The complete input sequence is:

[0096] 4.4 Encoder Encoding: The encoder E1 part consists of L stacked coding blocks. Each coding block first enters the normalization layer to output a normalized label sequence. The normalized label sequence then enters the multi-head self-attention layer. In the self-attention layer, the input sequence X is first subjected to three linear transformations to obtain the sequence of query Q, key K, and value V, which can be obtained through matrix multiplication and projection matrix W. Q W K W V To obtain:

[0097]

[0098] Among them W Q W K W V It is a weight matrix obtained through training. The obtained sequence of query Q, key K, and value V is divided into h heads, resulting in head q. i ,k i,v i This is a subset of the original sequence. An attention score is calculated for each head i using a dot product attention mechanism:

[0099]

[0100] in It is the scaling factor, d k This represents the dimensions of query Q and key K in each head. The Softmax function converts the attention scores into weights. Concatenating the attention outputs of each head yields the final output of the multi-head self-attention mechanism.

[0101] MultiHead(Q,K,V)=Concat(Attention(q1,k1,v1),Attention(q2,k2,v2),...,Attention(q h ,k h ,v h Each coding block contains a multi-head self-attention layer and a feedforward network (FFN). The operation of the l-th layer is as follows:

[0102] Z′ l =MultiHead(LayerNorm(Z) l-1 ))+Z l-1 ;

[0103] Z l =FFN(LayerNorm(Z′) l ))+Z′ l ;

[0104] Z l-1 Given the input sequence, LayerNorm is a layer normalization operation. Multi-head self-attention maintains the same output dimension. FFN is defined as:

[0105] FFN(U)=GELU(UW1+b1)W2+b2;

[0106] Where W and b are the trainable weight matrix and bias matrix, respectively. The output after multi-head self-attention concatenation undergoes a linear transformation through the trainable weight matrix W, and is then residually concatenated with the input sequence to finally output Z. l It includes global features ([CLS] position) and local features (positions of each block);

[0107] 4.5 Decoder Decoding: After E1 encoding by the encoder, the output feature matrix Z is completed. l Let the set of mask positions be... From Z l Extract mask block features:

[0108]

[0109] Then, the visual marker probability distribution of the mask block is predicted using decoder D:

[0110]

[0111] Where W is the learnable weight matrix, K is the size of the visual tag codebook, and y i For real visual marking;

[0112] The pre-training task uses masked image modeling, with the objective being to model based on masked block sequences. Reconstruct its visual token y i The loss function used is the cross-entropy function:

[0113]

[0114] Where y i It is the true visual marker of the i-th mask block. It is the final encoded feature of the i-th mask block. The optimization objective is to minimize the predicted negative log-likelihood of all mask blocks, that is, to maximize the labeled predicted likelihood of the mask block.

[0115] like Figure 8 The diagram shows the specific structure of the word segmenter T1, masker M, encoder E1, and decoder D. The model receives 224×224×3 image data and divides the input image into 16×16 image blocks. The feature dimension of the input image block after positional embedding is 768. The encoder has 12 layers N, with 12 attention heads h in the multi-head self-attention mechanism, and the masking ratio is 40%. The decoder has 1 layer and 12 attention heads. The feature dimension expansion ratio of the hidden layer to the embedding layer in the multilayer perceptron is 4, and the normalization layer type is layer normalization. The encoder maps the image blocks of the input image to the embedding space, and then the decoder attempts to reconstruct the true visual representation from the masked embedding representation.

[0116] The 3000 preprocessed unlabeled spatiotemporal images are transformed into a 224×224×3 format and input into the pre-trained network. A masking operation is performed using a masker M. The masked data is then input into the encoder E1, which outputs the reconstructed unlabeled data. The above steps are repeated until the masked unlabeled data gradually approximates the true unlabeled data.

[0117] During pre-training, the batch size was 16, the AdamW optimizer was used, the learning rate was 0.0001, the first moment exponential decay rate was 0.9, the second moment exponential decay rate was 0.999, the learning rate warm-up period was 5, and after 1600 iterations, the model loss function converged to 0.079, pre-training was stopped, and the model weight parameters were saved.

[0118] After the pre-trained network loss function meets the convergence condition, the training weight parameters of the E1 encoder are synchronized with those of the E2 encoder. Synchronization is achieved by loading the E1 encoder weight parameters during the initialization of the E2 encoder.

[0119] Step 5: Construct a downstream classification task network. Use a small amount of labeled data to learn the classification of downstream tasks and obtain new weights. Use the obtained new weights to complete the recognition and classification of unlabeled data.

[0120] Step 5 includes the following sub-steps:

[0121] 5.1 The initial weight parameters of the encoder E2 constructed by the downstream task are the same as the weight parameters of the encoder E1 of the pre-training task after training.

[0122] 5.2 In the position encoding step, the data tag value cls is added according to the tag data, the 0 position information is embedded, and it is concatenated with the tag sequence output by linear projection and input into the encoder output with the tag data;

[0123] 5.3. Use the tag value represented by cls as the final output of encoder E2;

[0124] 5.4 Calculate the cross-entropy loss function between the encoder output and the original data label values:

[0125]

[0126] Where y c For one-hot encoding of the real label, p c To predict probabilities.

[0127] The downstream task classification network in this embodiment is encoder E2, which outputs predicted pattern labels.

[0128] like Figure 9The diagram shows the specific structure of encoder E2 and linear classification head C. The model receives 224x224x3 image data, with the input image divided into 16x16 image patches. The feature dimension of the input image after location embedding is 768. The encoder has 12 layers N, including 12 attention heads h in the multi-head self-attention mechanism. The feature dimension expansion ratio of the hidden layers to the embedding layers in the multilayer perceptron is 4. The encoder maps the image patches and labels of the input image to the embedding space, extracts the embedding representations corresponding to the labels, normalizes them, and outputs the predicted label probability. The label dataset is input into the E2 encoder, whose training parameters are synchronized with those of the E1 encoder, to output the predicted pattern labels.

[0129] The 200 preprocessed labeled spatiotemporal images were transformed into a 224×224×3 format and input into the prediction training network. Label 'A' represents the gait events of individual experimenter A, label 'B' represents the gait events of individual experimenter B, label 'C' represents the gait events of individual experimenter C, and label 'D' represents the gait events of individual experimenter D.

[0130] Repeat the above steps to gradually approximate the predicted pattern labels with the true pattern labels, thereby obtaining a high-accuracy pattern recognition network model. During downstream task training, the training set to validation set ratio is 4:1, the batch size is 16, the AdamW optimizer is used, the learning rate is 0.0001, the first-order moment exponential decay rate is 0.9, the second-order moment exponential decay rate is 0.99, and cosine annealing is used for learning rate scheduling.

[0131]

[0132] Where x is the number of previous iterations, epochs is the number of training epochs, and lrf is the minimum value of the cosine function. This cosine annealing learning rate scheduling gradually decreases the learning rate during training, which helps the model converge better. In the early stages of training, a larger learning rate helps the model learn the features of the data more quickly; in the later stages of training, the learning rate gradually decreases, allowing the model to finely adjust its parameters and improve generalization performance. After 200 iterations, the loss function converged to 0.05326.

[0133] After the downstream classification task is trained to full convergence, the encoder E2 weights are obtained. All parameters of the downstream task network are frozen, and classification tests are performed on sufficient unlabeled data. 100 spatiotemporal graphs for each event are collected as the test set to test the model performance, and a confusion matrix is ​​plotted as follows: Figure 11 As shown, its F1 score reached 0.982. The formula for the F1 score is as follows:

[0134]

[0135] Where Precision represents accuracy, Recall represents recall, TP is the number of samples correctly predicted as positive, FP is the number of samples incorrectly predicted as positive, and FN is the number of samples incorrectly predicted as negative. The test results demonstrate that in downstream tasks, 50 labeled data points for each category can guarantee classification accuracy while ensuring training efficiency for the downstream task.

[0136] To train a pattern recognition model with the same structure, we pre-trained the BEiT pre-trained network (targeting a pre-trained encoder) with the same number of 3000 unlabeled images as input. After 800 iterations, the pre-trained weights were obtained. The pre-trained model weights were then synchronized, and the accuracy was measured using the same 20, 30, 40, 50, 60, 70, and 80 labeled images for each type after 200 iterations. The results are as follows: Figure 10 As shown in the figure, the test results demonstrate that in downstream tasks, only 50 labeled images of each type are needed to ensure classification accuracy while maintaining training efficiency for the downstream tasks. Figure 11 The diagram shows the confusion matrix of gait signals from four different individuals during the test.

[0137] The above provides a detailed description of the gait identification method based on the generative self-supervised visual pre-training model BEiT and DAS technology described in this invention. The examples above are only for the purpose of helping to understand the method and core ideas of this invention and are not intended to limit it. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of this invention should be considered equivalent substitutions and are included within the protection scope of this invention.

Claims

1. A gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology, characterized in that, Includes the following steps: Step 1, Scene Setup: Set up a distributed fiber optic vibration sensing system on the back of the carpet. Arrange optical fibers at certain intervals on the back of the carpet of a set size and connect the optical fibers to the distributed fiber optic sensing system. Step 2, Data Preparation: Different participants walked randomly on the carpet, and a distributed fiber optic vibration sensing system was set up on the carpet to collect gait signals of the participants walking freely. Step 3, Signal Preprocessing: Collect unlabeled and labeled data, extract data signals for signal preprocessing, and construct a time-frequency feature dataset; Step 4: Construct a pre-trained image reconstruction network for the self-supervised visual pre-trained model BEiT. The pre-trained network includes a word segmenter T1, a masker M, an encoder E1, and a decoder D. It extracts features from the input image and uses the above deep learning network to construct a feature extractor based on the self-supervised visual pre-trained model BEiT. It is then trained using unlabeled spatiotemporal data from the experimental scenario to obtain the weight parameters. Step 5: Construct a downstream classification task network. Use a small amount of labeled data to learn the classification of downstream tasks and obtain new weights. Use the obtained new weights to complete the identification and classification of unlabeled data.

2. The gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology as described in claim 1, characterized in that, Step 3 includes the following sub-steps: 3.1 Based on the data signal transmitted through the optical fiber, wavelet filtering is used to reduce noise in the signal and minimize the impact of low frequencies on the intrusion signal; 3.2 The distributed fiber optic vibration sensing system returns the original signal trajectory distributed in space at each moment, continuously accumulates the original signal on the time axis, performs Fourier transform on the signal, and constructs a time and frequency signal matrix; 3.

3. Through data processing methods, the obtained spatiotemporal signal matrix is ​​transformed into a 224×224 spatiotemporal graph, resulting in spatiotemporal graphs of four types of gait events in each scenario, with 2000 samples of each type of event in each scenario. 3.4 For data in laboratory scenarios, half of the data is used as pre-training data, and the other half is used as the validation set after downstream training.

3. A gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology as described in claim 1 or 2, characterized in that, Step 4 includes the following steps: 4.1 Image Block Processing: The spatiotemporal image of the DAS data is divided into fixed-size image blocks and flattened to obtain a block sequence. in Let be the flattened vector of the i-th block of the labeled sequence; 4.2 Random Masking by the Masker: The masker M randomly and uniformly selects a portion of the image patch as the center point according to the corresponding label group. Based on each center patch, it expands into a continuous square region of fixed size (e.g., 3×3). This region is then masked. The mask is represented by a learnable label sequence. The masking ratio needs to be moderate to reduce information redundancy between image patches, which is more conducive to the model network learning the global features of the image. The masked sequence is denoted as... Simultaneously, a pre-trained image tagger, i.e., a word segmenter T1, is introduced, typically a Discrete Variational Autoencoder (dVAE), to segment image patches. Quantized into discrete visual markers y i ∈{1, ...,K}, where K is the codebook size, and the tokenization process is as follows: in It is a discrete codebook.

4. The gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology as described in claim 3, characterized in that, Step 4 also includes the following steps: 4.3 Linear Projection and Position Encoding: The masked image patches are input into the linear projection layer. A learnable linear transformation maps the image patches to the latent space, and position encoding is added to preserve spatial information. A label corresponding to each image patch is generated, and an ordered label sequence with embedded position encoding is output, resulting in a label sequence with position encoding. i∈{1,2,…,N}; where E is a learnable projection matrix used to transform local features into high-level semantics. This is a location embedding vector used to encode spatial location information. The complete input sequence is: 4.4 Encoder Encoding: The encoder E1 part consists of L stacked coding blocks. Each coding block first enters the normalization layer to output the normalized label sequence. The normalized label sequence then enters the multi-head self-attention layer. In the self-attention layer, the input sequence X is first subjected to three linear transformations to obtain the sequence of query Q, key K, and value V, which can be obtained through matrix multiplication and projection matrix W. Q W K W V To obtain: Among them W Q W K W V The weight matrix obtained through training is used to divide the sequence of query Q, key K, and value V into h heads, resulting in head q. i ,k i ,v i Given a subset of the original sequence, we calculate the attention score for each head i using a dot product attention mechanism: in It is the scaling factor, d k The dimensions are the query Q and key K in each head. The Softmax function converts the attention scores into weights, and concatenates the attention outputs of each head to obtain the final output of multi-head self-attention: MultiHead(Q,K,V)=Concat(Attention(q1,k1,v1),Attention(q2,k2,v2),...,Attention(q h ,k h ,v h Each coding block contains a multi-head self-attention layer and a feedforward network (FFN). The operation of the l-th layer is as follows: WITH' l =MultiHead(LayerNorm(Z l-1 ))+Z l-1 ; Z l =FFN(LayerNorm(Z′ l ))+Z′ l ; Z l-1 Given the input sequence, LayerNorm is a layer normalization operation. Multi-head self-attention maintains the same output dimension. FFN is defined as follows: FFN(U)=GELU(UW1+b1)W2+b2; Where W and b are the trainable weight matrix and bias matrix, respectively. The output after multi-head self-attention concatenation undergoes a linear transformation through the trainable weight matrix W, and is then residually concatenated with the input sequence to finally output Z. l It includes global features, namely the [CLS] position, and local features, namely the positions of each block.

5. The gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology as described in claim 4, characterized in that, Step 4 also includes the following steps: 4.5 Decoder Decoding: After E1 encoding by the encoder, the output feature matrix Z is completed. l Let the set of mask positions be... From Z l Extract mask block features: Then, the visual marker probability distribution of the mask block is predicted using decoder D: Where W is the learnable weight matrix, K is the size of the visual tag codebook, and y i For real visual marking; The pre-training task uses masked image modeling, with the objective being to model based on masked block sequences. Reconstruct its visual token y i The loss function used is the cross-entropy function: Where y i It is the true visual marker of the i-th mask block. It is the final encoded feature of the i-th mask block. The optimization objective is to minimize the predicted negative log-likelihood of all mask blocks, that is, to maximize the labeled predicted likelihood of the mask block.

6. A gait identification method based on the generative self-supervised visual pre-trained model BEiT and DAS technology as described in claim 1 or 2, characterized in that, Step 5 includes the following sub-steps: 5.1 The initial weight parameters of the encoder E2 constructed by the downstream task are the same as the weight parameters of the encoder E1 of the pre-training task after training. 5.2 In the position encoding step, the data tag value cls is added according to the tag data, the 0 position information is embedded, and it is concatenated with the tag sequence output by linear projection and input into the encoder output with the tag data; 5.

3. Use the tag value represented by cls as the final output of encoder E2; 5.4 Calculate the cross-entropy loss function between the encoder output and the original data tag values: Where y c For one-hot encoding of the real label, p c To predict probabilities.

Images