Gait micro-Doppler spectrogram generation system, method and computer program product

Abstract

The embodiment of the invention discloses a gait micro-Doppler spectrogram generation system and method and a computer program product. Efficient and controllable generation of a gait micro-Doppler spectrogram can be achieved. The system comprises a generation module and a reconstruction module, the generation module is used for responding to received random Gaussian noise and labels of samples needing to be generated, generating a model by utilizing pre-trained potential condition diffusion, and reconstructing the random Gaussian noise and the labels of the samples needing to be generated according to the random Gaussian noise and the labels of the samples needing to be generated. Generating a potential vector representation conforming to a label of a sample needing to be generated, and inputting the potential vector representation into the reconstruction module; and the reconstruction module is used for decoding the potential vector representation into a gait micro-Doppler spectrogram in a pixel space by utilizing a decoder in a pre-trained auto-encoder, and the auto-encoder is trained by adopting an adversarial learning mechanism based on a pre-configured discriminator.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a gait micro-Doppler spectrum generation system, method, and computer program product. Background Technology

[0002] Gait micro-Doppler spectra are a type of image-based gait feature. Currently, deep learning-based gait recognition methods are commonly used to mine inherent features representing user identity in gait micro-Doppler spectra in order to achieve user identification.

[0003] The performance of deep learning-based gait recognition methods is constrained by the quality (i.e., diversity) and size of the gait dataset. A large-scale and diverse gait dataset can help break through the performance ceiling of gait recognition and the limitations of application scenarios.

[0004] In related technologies, gait data (i.e., gait micro-Doppler spectra) are mainly collected from each user individually to construct a gait dataset. This method is inefficient, and gait data collection requires not only the deployment of specialized hardware but also fixed environmental conditions (such as the user needing to walk along a predefined walking path or approximate direction) to facilitate data collection. Furthermore, the gait micro-Doppler spectra that can be collected are limited by the user's gait pattern, making it difficult to efficiently obtain large-scale and highly diverse gait micro-Doppler spectra, thereby limiting the quality and scale of the constructed gait dataset. Summary of the Invention

[0005] The purpose of this application is to provide a gait micro-Doppler spectrum generation system, method, and computer program product that can achieve efficient and controllable generation of gait micro-Doppler spectra, thereby facilitating the efficient acquisition of large-scale and highly diverse gait micro-Doppler spectra.

[0006] In a first aspect, embodiments of this application provide a gait micro-Doppler spectrum generation system, the system comprising a generation module and a reconstruction module, wherein: The generation module is configured to, in response to receiving random Gaussian noise and the label of the desired generated sample, use a pre-trained latent conditional diffusion generation model to generate a latent vector representation that conforms to the label of the desired generated sample based on the random Gaussian noise and the label of the desired generated sample, and input the latent vector representation to the reconstruction module; The reconstruction module is used to decode the latent vector representation into a gait micro-Doppler spectrum in pixel space using the decoder in the pre-trained autoencoder. The autoencoder includes at least an encoder and a decoder. The encoder is used to compress a high-dimensional gait microDoppler spectrum into a low-dimensional latent vector representation. The autoencoder is trained using an adversarial learning mechanism based on a pre-configured discriminator. The latent conditional diffusion generation model uses the labels corresponding to the gait samples and the encoder in the pre-trained autoencoder to train the latent vector representation extracted from the gait samples.

[0007] Optionally, the autoencoder further includes a vector quantization layer, which is deployed in the decoder, wherein: The vector quantization layer is used to quantize the continuous latent vector representations input to the decoder, so that the decoder can more effectively capture semantic information in the gait microDoppler spectrogram and better control the diversity of generated images during training.

[0008] Optionally, the computation process performed by the vector quantization layer is represented as follows:

[0009] in, express The corresponding quantized vector, codebook The first in Discrete elements Let represent the vector at position (i, j) in a continuous latent vector representation.

[0010] Optionally, the loss calculation formula for the autoencoder is expressed as follows:

[0011] in, This represents the loss of the autoencoder; This represents the L2 loss, used to evaluate the quality of the gait samples reconstructed by the decoder. Let represent the discrete regularization loss, and:

[0012] in, This indicates a gradient stopping operation, which occurs during forward propagation. The value in remains unchanged during loss backpropagation. The value in the middle is differentiated with respect to 0; This represents the weight hyperparameter; This represents the quantized latent vector representation; Indicates encoder For the input gait samples The output is a continuous latent vector representation.

[0013] Optionally, the discriminator is configured to perform the following steps during training using an adversarial learning mechanism: The gait samples reconstructed by the decoder are divided into small patches; By independently determining whether each patch is real data, the information loss of the original gait samples during image compression is detected, which helps the autoencoder learn how to preserve important information as much as possible during image compression.

[0014] Optionally, the adversarial loss used in the training process employing the adversarial learning mechanism is determined by the following formula:

[0015] in, Indicating resistance to loss, This represents the discriminator. This represents the input gait sample. This represents the reconstructed gait sample.

[0016] Optionally, during the training process using an adversarial learning mechanism, the decoder in the autoencoder and the discriminator are trained alternately in a loop based on a preset training count ratio until the decoder in the autoencoder meets a preset training termination condition and the training ends.

[0017] Optionally, the generation module further utilizes a conditional encoder to convert the label into a category conditional embedding, so that the latent conditional diffusion generation model generates a latent vector representation that conforms to the label based on the category conditional embedding; The processing steps required for the latent conditional diffusion generation model during training include: In the forward diffusion stage of the latent conditional diffusion generation model, noise is added to the latent vector representation corresponding to the gait sample to gradually approach Gaussian noise; In the reverse denoising stage of the latent conditional diffusion generation model, the latent vector representation after denoising is denoised according to the category conditional embedding corresponding to the gait sample, so as to obtain the latent vector representation before denoising.

[0018] A second aspect of this application provides a method for generating gait micro-Doppler spectra, the method comprising: Random Gaussian noise and the labels of the samples to be generated are input into the gait micro-Doppler spectrum generation system as described in the first aspect to obtain gait micro-Doppler spectra that match the labels.

[0019] A third aspect of this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the gait microDoppler spectrum generation method as described in the second aspect.

[0020] A fourth aspect of this application provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the gait microDoppler spectrum generation method as described in the second aspect.

[0021] A fifth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the gait micro-Doppler spectrum generation method as described in the second aspect.

[0022] As can be seen from the above technical solutions, this application introduces a latent conditional diffusion generation model and uses labels to guide the generation direction of the latent conditional diffusion generation model. This enables controllable generation of gait micro-Doppler spectra and avoids the limitations imposed by hardware devices, fixed environmental conditions, and user gait patterns in gait micro-Doppler spectra acquisition. Furthermore, this application utilizes deep learning technology to compress the high-dimensional gait micro-Doppler spectra to reduce the feature dimensionality, allowing gait features to diffuse and generate in a low-dimensional latent space. This reduces computational resource consumption, accelerates the inference process, and overcomes the limitations of relying solely on latent conditional diffusion generation models. This application addresses the problems of high computational cost and slow inference speed in generating gait micro-Doppler spectra using traditional methods. Furthermore, it introduces an adversarial learning mechanism into the deep learning-based image compression process to suppress information loss during compression, thereby mitigating the information loss problem caused by image compression. As a result, this application enables efficient and controllable generation of gait micro-Doppler spectra, facilitating the efficient acquisition of large-scale and highly diverse gait micro-Doppler spectra. It also eliminates the costs of data acquisition, processing, and annotation during data construction, thus providing a technical foundation for building high-quality, large-scale, and low-cost gait datasets. Attached Figure Description

[0023] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic diagram of a gait microDoppler spectrum generation system provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of a self-encoder provided in an embodiment of this application; Figure 3 A schematic diagram illustrating a latent feature quantification process provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the processing operations performed by a generation module according to an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0026] In the field of wireless sensing, gait recognition methods based on millimeter-wave radar have demonstrated feature extraction capabilities that surpass traditional signal processing. Specifically, they fully exploit the inherent features representing user identity in the gait micro-Doppler spectrum by using deep learning technology, thereby achieving accurate user identification.

[0027] Since gait recognition methods are mainly based on deep learning technology, their performance is limited by the quality and scale of the gait dataset. A large-scale and diverse gait dataset can help break through the performance ceiling of gait recognition and the limitations of application scenarios, but the process of constructing the dataset itself is a huge challenge.

[0028] From an objective environmental perspective, collecting gait data (also known as millimeter-wave gait data, millimeter-wave human gait data, i.e. gait micro-Doppler spectrum) from users one by one is inefficient. Moreover, gait data collection not only requires the deployment of specialized hardware equipment, but also requires fixed environmental conditions (such as users needing to walk along a predefined walking path or in an approximate direction) to facilitate data collection. Such fixed environmental conditions severely restrict the generalization ability of the gait recognition system.

[0029] From a subjective human perspective, the gait data that can be collected is also limited by the user's gait pattern. Gait pattern is a direct expression of a person's dynamic walking characteristics and is influenced by age, physiological state, psychological state, occupation, and other attributes.

[0030] Therefore, the current construction of gait datasets lacks an efficient and effective scheme for obtaining large-scale and highly diverse gait micro-Doppler spectra. As a result, constructing a large-scale gait dataset with high data diversity is a difficult task, which is not conducive to the rapid development of related scientific research and urgently needs to be solved.

[0031] Based on the above analysis, and addressing the problem that related technologies struggle to efficiently obtain large-scale and highly diverse gait micro-Doppler spectra, this application provides a gait micro-Doppler spectrum generation system, method, and computer program product. This system can efficiently and controllably generate gait micro-Doppler spectra based on a latent conditional diffusion generation model, thereby facilitating the efficient acquisition of large-scale and highly diverse gait micro-Doppler spectra.

[0032] See Figure 1 The diagram shown is a structural schematic of a gait micro-Doppler spectrum generation system provided in an embodiment of this application. The system includes a generation module and a reconstruction module, wherein: The generation module is configured to, in response to receiving random Gaussian noise and the label of the desired generated sample, use a pre-trained latent conditional diffusion generation model to generate a latent vector representation that conforms to the label of the desired generated sample based on the random Gaussian noise and the label of the desired generated sample, and input the latent vector representation to the reconstruction module; The reconstruction module is used to decode the latent vector representation into a gait micro-Doppler spectrum in pixel space using the decoder in the pre-trained autoencoder. The autoencoder includes at least an encoder and a decoder. The encoder is used to compress a high-dimensional gait microDoppler spectrum into a low-dimensional latent vector representation. The autoencoder is trained using an adversarial learning mechanism based on a pre-configured discriminator. The latent conditional diffusion generation model uses the labels corresponding to the gait samples and the encoder in the pre-trained autoencoder to train the latent vector representation extracted from the gait samples.

[0033] It should be noted that the input noise of the latent conditional diffusion generative model during the inference phase is pure random Gaussian noise, while during the training phase, the noise obtained by adding Gaussian noise to the latent vector output by the encoder in the autoencoder (to make it conform to a Gaussian distribution) is used as the training input.

[0034] In this embodiment, the present application uses an autoencoder as the basic architecture and introduces an adversarial learning mechanism using a discriminator during the training process of the autoencoder, thereby constructing a reconstruction module based on adversarial learning. At the same time, the present application also constructs a generation module based on a latent conditional diffusion generation model, which can generate a gait latent representation (i.e., latent vector representation) of a specified category according to random Gaussian noise and the labels of the samples to be generated (such as user-related category labels, where different labels correspond to different users). This latent gait representation is then decoded by the decoder in the reconstruction module into a gait micro-Doppler spectrum in pixel space, thereby obtaining a gait micro-Doppler spectrum that conforms to the input label.

[0035] Specifically, when constructing the system, a pre-established dataset and discriminator are first used, and an adversarial learning mechanism is employed to train the autoencoder. During training, the discriminator is mainly used to analyze the local details of the gait samples (i.e., gait micro-Doppler spectra) reconstructed by the decoder, thereby detecting the loss of sample information during image compression to ensure that important information is preserved; the autoencoder is mainly used to learn how to preserve important information as much as possible during image compression. Optionally, the dataset can be established through the following steps: Collect millimeter-wave human gait data; The collected raw data is processed (e.g., noise reduction, static clutter elimination, Fourier transform, etc.). Gait micro-Doppler spectra (i.e., gait samples) are extracted from the processed data to construct a dataset.

[0036] Then, labeled gait samples are input into a trained autoencoder to extract latent vector representations of the gait samples through the encoder. The latent vector representations extracted by the encoder and their corresponding labels are then used to train a latent conditional diffusion generation model. A reconstruction module is then constructed based on the decoder in the trained autoencoder, and a generation module is constructed based on the latent conditional diffusion generation model, thus obtaining the gait micro-Doppler spectrum generation system.

[0037] Optionally, during the compression and decompression of gait samples, L2 loss can be used to evaluate the quality of the reconstructed samples:

[0038] in, For L2 loss, This represents the input gait sample. This represents the reconstructed gait sample.

[0039] When using the system, random Gaussian noise and the labels of the samples to be generated are input into the gait micro-Doppler spectrum generation system. The generation module in the system will input the random Gaussian noise and the labels of the samples to be generated (such as the category label corresponding to a user) into the latent conditional diffusion generation model to generate a latent vector representation that matches the label of the sample to be generated (such as generating a latent vector representation corresponding to the user). Then, the generation module will input the generated latent vector representation into the decoder in the reconstruction module so that the decoder can decode the latent vector representation into a gait micro-Doppler spectrum in pixel space.

[0040] It should be noted that this application takes into account that gait micro-Doppler spectra are gait features in image format. Therefore, a latent conditional diffusion-based generative model can be used to generate gait micro-Doppler spectra, but the following two issues need to be addressed: (1) Gait micro-Doppler spectra have a high-dimensional data structure. Each pixel contains a large amount of feature information. High-dimensional data processing usually requires more computational operations. Therefore, when reasoning and generating in pixel space, it will bring huge computational consumption and the reasoning speed will be greatly limited.

[0041] To address the aforementioned issues, this application utilizes deep learning technology to compress high-dimensional image features, thereby reducing feature dimensionality, minimizing computational resource consumption, and accelerating the inference process.

[0042] (2) Diffusion and generation in a low-dimensional potential space will reduce computational costs, but data compression (i.e. image compression) will lead to information loss. How to maintain the accuracy of information and reduce information loss while compressing data is a difficult problem.

[0043] To address the aforementioned issues, this application introduces an adversarial learning mechanism into deep learning-based image compression to suppress information loss during the data compression process.

[0044] Therefore, in order to solve the problem of constructing high-quality, large-scale gait datasets and the two problems mentioned above, this application designs a gait micro-Doppler spectrum generation system using a latent conditional diffusion model, which can achieve efficient and controllable generation of gait micro-Doppler spectra and effectively reduce the cost of dataset construction.

[0045] As can be seen from the above technical solutions, this application introduces a latent conditional diffusion generation model and uses labels to guide the generation direction of the latent conditional diffusion generation model. This enables controllable generation of gait micro-Doppler spectra and avoids the limitations imposed by hardware devices, fixed environmental conditions, and user gait patterns in gait micro-Doppler spectra acquisition. Furthermore, this application utilizes deep learning technology to compress the high-dimensional gait micro-Doppler spectra to reduce the feature dimensionality, allowing gait features to diffuse and generate in a low-dimensional latent space. This reduces computational resource consumption, accelerates the inference process, and overcomes the limitations of relying solely on latent conditional diffusion generation models. This application addresses the problems of high computational cost and slow inference speed in generating gait micro-Doppler spectra using traditional methods. Furthermore, it introduces an adversarial learning mechanism into the deep learning-based image compression process to suppress information loss during compression, thereby mitigating the information loss problem caused by image compression. As a result, this application enables efficient and controllable generation of gait micro-Doppler spectra, facilitating the efficient acquisition of large-scale and highly diverse gait micro-Doppler spectra. It also eliminates the costs of data acquisition, processing, and annotation during data construction, thus providing a technical foundation for building high-quality, large-scale, and low-cost gait datasets.

[0046] In one alternative embodiment, refer to Figure 2 The diagram shown illustrates the structure of the autoencoder, which further includes a vector quantization layer deployed within the decoder. The vector quantization layer is used to quantize the continuous latent vector representations input to the decoder, so that the decoder can more effectively capture semantic information in the gait microDoppler spectrogram and better control the diversity of generated images during training.

[0047] It should be noted that during training, the features output by the encoder (i.e., the continuous latent vector representation) serve as input to the decoder and, after being denoised, as input samples for the inverse denoising stage to train the latent conditional diffusion generative model. During this training process, the decoder first quantizes the encoder's output through a vector quantization layer before inputting it into the subsequent network, thereby enabling the decoder to effectively capture rich semantic information and better control the generation of gait images.

[0048] Accordingly, during the generation phase, the continuous latent vector representation generated by the latent conditional diffusion generation model is input to the decoder. The decoder first quantizes it through a vector quantization layer, and then decodes the quantized latent vector representation into a gait micro-Doppler spectrum in pixel space.

[0049] In this embodiment, considering the strong dependence of the autoencoder on training samples, the decoder is accustomed to obtaining compressed data only from the compressed representation of the training samples by the encoder, rather than randomly generated compressed data. To avoid overfitting of the autoencoder on the training samples and to meet the requirement of a high-quality, rich gait latent space, this application adds a vector quantization layer to the decoder to map the continuous latent vector representation input to the decoder to a discrete representation space. Thus, for a given gait sample... decoder During training, it can be used according to The reconstructed sample Approximating it, where, This represents the quantized latent vector representation.

[0050] Optionally, the vector quantization layer is applied to the encoder. The output features (i.e., the latent vector representation) Quantization is performed one by one, specifically in the codebook. Find the closest match in a set of discrete elements of a fixed size. The vector at position (i, j) of As the corresponding quantized vector .

[0051] For example, refer to Figure 3 The diagram illustrates the latent feature quantization process. The computation process performed by the vector quantization layer can be represented as follows:

[0052] in, express The corresponding quantized vector, codebook The first in Discrete elements Let represent the vector at position (i, j) in a continuous latent vector representation. This indicates a vector quantization layer. Indicates encoder, This represents the input gait sample. Indicates encoder The output is a continuous latent vector representation.

[0053] In practice, the vector quantization layer first calculates the features. Find the distances to K vectors in the codebook (K represents the number of vectors in the codebook), then find the minimum value of the distance array, and return the index of the vector corresponding to the minimum distance (i.e., the index of the vector in the codebook that is closest to the minimum distance). The index of the nearest vector is used to retrieve the vector from the codebook, thus obtaining... .

[0054] Optionally, considering that vector quantization is a non-differentiable operation, and network errors cannot be propagated from the decoder to the encoder during backpropagation, this application introduces a pass-through gradient estimation technique to complete gradient replication from the decoder to the encoder, thereby solving the aforementioned problem of non-differentiability of quantization. Specifically, after introducing the vector quantization layer, the loss calculation formula of the autoencoder can be expressed as follows:

[0055] in, This represents the loss of the autoencoder; This represents the L2 loss, used to evaluate the quality of the gait samples reconstructed by the decoder. Let represent the discrete regularization loss, and:

[0056] in, This indicates a gradient stopping operation, which occurs during forward propagation. The value in remains unchanged during loss backpropagation. The value in the expression is differentiated with respect to 0, meaning there is no gradient in this calculation; This represents the weight hyperparameter; This represents the quantized latent vector representation; Indicates encoder For the input gait samples The output is a continuous latent vector representation.

[0057] For example, refer to Figure 4 The diagram illustrates the processing operations performed by the generation module. The processing mainly includes: the generation module using a conditional encoder (such as the nn.Embedding module in PyTorch) to convert labels (such as user category numbers) into category conditional embeddings; then, the generation module uses the encoder in the autoencoder... The output latent vector representation, the class conditional embedding (used to introduce sample label information in the inverse denoising stage to guide the generation direction of the latent conditional diffusion generative model), and the diffusion step size t are input into the latent conditional diffusion generative model so that the latent conditional diffusion generative model performs the processing steps involved in the training process, that is, to obtain the noisy latent vector representation in the inverse denoising stage (i.e., generate random noise). And generate latent vector representations in the reverse denoising stage. .

[0058] Optionally, after introducing sample information in the reverse denoising stage, the loss function of the conditional diffusion generation model is... It can be represented as follows:

[0059] in, Indicates the expected value. This represents the latent vector representation of the encoder output. Represents the noise vector. This represents the noise in the prediction. Indicates the diffusion step size. Let represent the noisy latent vector representation obtained at a diffusion step size t. Labels representing gait samples, This indicates a condition encoder.

[0060] Through joint optimization and The generation module based on the latent conditional diffusion generation model can generate gait latent representations of a specified category according to the conditional information (i.e., labels), and decode the generated gait latent representations into gait micro-Doppler spectra in pixel space through the decoder of the autoencoder.

[0061] It is understandable that the gait microDoppler spectrum generation scheme provided in this application mainly includes two stages: In the first stage, the autoencoder (trained using an adversarial learning mechanism) reconstructs gait samples. During the reconstruction process, the encoder is responsible for extracting the latent space of the gait samples.

[0062] In the second stage, the latent conditional diffusion generation model generates gait latent representations (i.e., latent vector representations) in the latent space of gait; where, for example... Figure 4 As shown, the U-Net architecture model is mainly used to predict noise in the reverse denoising stage of the latent conditional diffusion generative model. The U-Net architecture model mainly includes a ResBlock module (i.e., residual block) and an attention module; the latent conditional diffusion generative model ends denoising after repeating the denoising process T times. Figure 4 The model has already been de-operated once and needs to be repeated (T-1) times. Then, the generated gait latent representation is decoded into a gait image (i.e., gait micro-Doppler spectrum) through the decoder of the autoencoder in the first stage (which has a vector quantization layer). Here, T represents the total step length (i.e., the maximum step length) in the latent conditional diffusion generation model.

[0063] In an optional embodiment, the discriminator is configured to perform the following steps during training using an adversarial learning mechanism: The gait samples reconstructed by the decoder are divided into patches. By independently determining whether each small patch is real data, the information loss of the original gait samples during image compression is detected, which helps the autoencoder learn how to preserve important information as much as possible during image compression.

[0064] In this embodiment, considering that the greater the compression ratio of an image during image compression, the more information will be lost, this application designs a patch-based discriminator and autoencoder adversarial training method to minimize information loss during image compression as much as possible, in order to ensure that important information is still preserved even under extreme compression.

[0065] Specifically, during adversarial training with the autoencoder, the patch-based discriminator divides the image (i.e., the reconstructed image sample) into multiple small patches. Each patch is judged independently, i.e., whether it is real data. Thus, the discriminator focuses on local details in the image to detect information loss during image compression, thereby ensuring that important information is preserved during image compression. In contrast, the autoencoder can focus on learning how to preserve important information as much as possible during compression.

[0066] Optionally, the adversarial loss used in the training process employing the adversarial learning mechanism is determined by the following formula:

[0067] in, Indicating resistance to loss, This represents the discriminator. This represents the input gait sample. This represents the reconstructed gait sample.

[0068] Optionally, during the training process using an adversarial learning mechanism, the decoder in the autoencoder and the discriminator are trained alternately in a loop based on a preset training count ratio until the decoder in the autoencoder meets a preset training termination condition and the training ends.

[0069] For example, after the decoder (or the autoencoder as a whole) in the autoencoder is trained 4 times, the discriminator is trained once, and the training is repeated in this ratio until the decoder (or the autoencoder as a whole) in the autoencoder converges.

[0070] Optionally, the generation module further utilizes a conditional encoder to convert the label into a category conditional embedding, so that the latent conditional diffusion generation model generates a latent vector representation that conforms to the label based on the category conditional embedding; The processing steps required for the latent conditional diffusion generation model during training include: In the forward diffusion stage of the latent conditional diffusion generation model, noise is added to the latent vector representation corresponding to the gait sample to gradually approach Gaussian noise; In the reverse denoising stage of the latent conditional diffusion generation model, the latent vector representation after denoising is denoised according to the category conditional embedding corresponding to the gait sample, so as to obtain the latent vector representation before denoising.

[0071] This application embodiment also provides a method for generating gait micro-Doppler spectra, the method comprising: Random Gaussian noise and the labels of the samples to be generated are input into the gait micro-Doppler spectrum generation system as described in the above system embodiment to obtain gait micro-Doppler spectra that match the labels.

[0072] This application also provides an electronic device, see embodiments thereof. Figure 5 , Figure 5 This is a schematic diagram of the electronic device proposed in an embodiment of this application. Figure 5 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus for communication. The memory 110 stores a computer program that can run on the processor 120 to implement the steps in the gait micro-Doppler spectrum generation method disclosed in the embodiments of this application.

[0073] This application also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the gait micro-Doppler spectrum generation method disclosed in this application.

[0074] This application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the gait micro-Doppler spectrum generation method disclosed in this application.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0076] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, systems, devices, storage media, and program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0081] The above provides a detailed description of the gait micro-Doppler spectrum generation system, method, and computer program product provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A gait micro-Doppler spectrum generation system, characterized in that, The system includes a generation module and a reconstruction module, wherein: The generation module is configured to, in response to receiving random Gaussian noise and the label of the desired generated sample, use a pre-trained latent conditional diffusion generation model to generate a latent vector representation that conforms to the label of the desired generated sample based on the random Gaussian noise and the label of the desired generated sample, and input the latent vector representation to the reconstruction module; The reconstruction module is used to decode the latent vector representation into a gait micro-Doppler spectrum in pixel space using the decoder in the pre-trained autoencoder. The autoencoder includes at least an encoder and a decoder. The encoder is used to compress a high-dimensional gait microDoppler spectrum into a low-dimensional latent vector representation. The autoencoder is trained using an adversarial learning mechanism based on a pre-configured discriminator. The latent conditional diffusion generation model uses the labels corresponding to the gait samples and the encoder in the pre-trained autoencoder to train the latent vector representation extracted from the gait samples.

2. The system according to claim 1, characterized in that, The autoencoder further includes a vector quantization layer, which is deployed in the decoder, wherein: The vector quantization layer is used to quantize the continuous latent vector representations input to the decoder, so that the decoder can more effectively capture semantic information in the gait microDoppler spectrogram and better control the diversity of generated images during training.

3. The system according to claim 2, characterized in that, The computation process performed by the vector quantization layer is represented as follows: in, express The corresponding quantized vector, codebook The first in Discrete elements Let represent the vector at position (i, j) in a continuous latent vector representation.

4. The system according to claim 2, characterized in that, The loss calculation formula for the autoencoder is expressed as follows: in, This represents the loss of the autoencoder; This represents the L2 loss, used to evaluate the quality of the gait samples reconstructed by the decoder. Let represent the discrete regularization loss, and: in, This indicates a gradient stopping operation, which occurs during forward propagation. The value in remains unchanged during loss backpropagation. The value in the interval is taken as a derivative with respect to 0; This represents the weight hyperparameter; This represents the quantized latent vector representation; Indicates encoder For the input gait samples The output is a continuous latent vector representation.

5. The system according to claim 1, characterized in that, The discriminator is used to perform the following steps during training using an adversarial learning mechanism: The gait samples reconstructed by the decoder are divided into small patches; By independently determining whether each patch is real data, the information loss of the original gait samples during image compression is detected, which helps the autoencoder learn how to preserve important information as much as possible during image compression.

6. The system according to claim 5, characterized in that, The adversarial loss used in the training process employing the adversarial learning mechanism is determined by the following formula: in, Indicating resistance to loss, This represents the discriminator. This represents the input gait sample. This represents the reconstructed gait sample.

7. The system according to any one of claims 1-6, characterized in that, During the training process using the adversarial learning mechanism, the decoder in the autoencoder and the discriminator are trained alternately in a cyclical manner based on a preset training count ratio until the decoder in the autoencoder meets the preset training termination condition and the training ends.

8. The system according to any one of claims 1-6, characterized in that, The generation module also uses a conditional encoder to convert the label into a category conditional embedding, so that the latent conditional diffusion generation model can generate a latent vector representation that conforms to the label based on the category conditional embedding. The processing steps required for the latent conditional diffusion generation model during training include: In the forward diffusion stage of the latent conditional diffusion generation model, noise is added to the latent vector representation corresponding to the gait sample to gradually approach Gaussian noise; In the reverse denoising stage of the latent conditional diffusion generation model, the latent vector representation after denoising is denoised according to the category conditional embedding corresponding to the gait sample, so as to obtain the latent vector representation before denoising.

9. A method for generating gait micro-Doppler spectra, characterized in that, The method includes: Random Gaussian noise and the labels of the samples to be generated are input into the gait micro-Doppler spectrum generation system as described in any one of claims 1 to 8 to obtain a gait micro-Doppler spectrum that matches the labels.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the gait microDoppler spectrum generation method as described in claim 9.

Images