A method and apparatus for radar scattering simulation of human motion driven by text instructions

Abstract

The application relates to a radar scattering simulation method and device for human body action driven by a text instruction. The method first fuses a semantic feature vector extracted from a text with an action prior vector to obtain a semantic conditional vector, automatically generates a human body skeleton action sequence based on the vector, combines a body shape parameter to construct a human body three-dimensional grid sequence, completes electromagnetic scattering calculation based on a radar parameter and the grid model to obtain a radar echo data cube, and finally realizes multi-dimensional visual imaging through signal processing. The application breaks through the disadvantage that a traditional simulation can only use a fixed action template, flexibly generates various complex human body actions by relying on text semantics combined with prior constraints, significantly improves semantic controllability and action diversity, realizes full-automatic integrated operation from natural language input to radar imaging through the whole link, and does not need manual intervention to design an action track and simulation parameters, and the practicability and engineering application value are outstanding.

Description

Technical Field

[0001] This application relates to the field of human body radar scattering simulation technology, and in particular to a text command-driven human body motion radar scattering simulation method and apparatus. Background Technology

[0002] In radar-based human behavior perception research, acquiring real radar data faces multiple obstacles. Core equipment such as millimeter-wave radar and ultra-wideband MIMO radar not only have high procurement costs, but their experimental deployment is also highly dependent on dedicated hardware platforms and fixed acquisition environments, limiting research activities to specific sites and resource conditions. Data acquisition tasks involving diverse human movement types, varying observation angles, and complex scene conditions require significant time and human resources for repeated experiments. Furthermore, stringent requirements are placed on the repeatability of movement execution, time synchronization accuracy, and annotation accuracy, significantly increasing the complexity and economic burden of data acquisition. In addition, external variables such as environmental noise interference, individual physiological differences, and object occlusion further weaken the stability and consistency of real-world data acquisition, making the construction of large-scale, high-quality radar human datasets with accurate spatiotemporal labels a major challenge.

[0003] Existing human radar scattering simulation schemes generally use preset motion templates, manually designed trajectory paths, or limited rule action libraries as a basis. The motion sequences generated by such methods are of a single type and lack flexibility. They cannot dynamically generate human motion processes with rich semantic connotations and complex constraints in response to natural language commands input by users, thus making it difficult to support the demand for diverse and highly adaptable simulation data in practical applications. Summary of the Invention

[0004] To achieve the above objectives, a first aspect of the present invention provides a text-instruction-driven human motion radar scattering simulation method, the method comprising: The semantic feature vector extracted from the text instruction is fused with the preset action prior vector to obtain the semantic condition vector; the text instruction includes action description information. Based on the semantic condition vector, a human skeleton motion sequence is generated; the human skeleton motion sequence includes the three-dimensional spatial position and rotation parameters of the human joints in each time frame; Based on the human skeletal movement sequence, a three-dimensional human mesh sequence is constructed; Based on the radar parameters and the three-dimensional mesh sequence of the human body, the radar echo data cube is calculated; Multidimensional imaging visualization is completed based on the radar echo data cube.

[0005] The text-command-driven human motion radar scattering simulation method provided in the first aspect of this application has at least the following beneficial effects: This method first fuses semantic feature vectors extracted from text with prior action vectors to obtain semantic condition vectors. Based on these vectors, a sequence of human skeletal movements is automatically generated. Then, combined with body shape parameters, a 3D human mesh sequence is constructed. Electromagnetic scattering calculations are performed based on radar parameters and the mesh model to obtain a radar echo data cube. Finally, multi-dimensional visualization imaging is achieved through signal processing. This method completely eliminates the limitations of traditional radar data acquisition, which relies on expensive hardware, dedicated facilities, and significant manpower, greatly reducing data acquisition costs and time. It can quickly produce standardized simulation data in batches. Simultaneously, it overcomes the drawback of traditional simulations that can only use fixed action templates. Relying on text semantics combined with prior constraints, it flexibly generates various complex human movements, significantly improving semantic controllability and movement diversity. The entire chain achieves fully automated integrated operation from natural language input to radar imaging, without the need for manual intervention in designing action trajectories and simulation parameters. The generated human movements closely resemble real motion patterns, and the 3D triangular mesh geometry is reasonable. The radar echoes obtained by combining physical optics methods and complex Gaussian noise simulation highly reproduce the real detection environment. The final output multi-dimensional imaging has outstanding practicality and engineering application value.

[0006] The second aspect of this application provides a text-command-driven human motion radar scattering simulation device, the device comprising: The semantic condition vector extraction module is used to fuse the semantic feature vector extracted from the text instruction with the preset action prior vector to obtain the semantic condition vector; the text instruction includes action description information. The human skeleton motion extraction module is used to generate a human skeleton motion sequence based on the semantic condition vector; the human skeleton motion sequence includes the three-dimensional spatial position and rotation parameters of the human joints in each time frame; The human body 3D mesh extraction module is used to construct a human body 3D mesh sequence based on the human skeletal movement sequence; The radar echo data extraction module is used to calculate the radar echo data cube based on radar parameters and the three-dimensional mesh sequence of the human body. The multidimensional visualization module is used to complete multidimensional imaging visualization based on the radar echo data cube.

[0007] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to execute the above-described text instruction-driven human motion radar scattering simulation method.

[0008] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described text-instruction-driven human motion radar scattering simulation method.

[0009] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of a text-instruction-driven human motion radar scattering simulation method provided in an embodiment of this application; Figure 2 This is a static human arm-out posture diagram generated based on the motion diffusion model, provided in an embodiment of this application. Figure 3(a) is a horizontal plane image of radar characterization of the arm extension posture of a simulated human body in SMPL provided in an embodiment of this application; Figure 3(b) is a coronal image of the radar characterization of the arm extension posture of the SMPL simulated human body provided in the embodiment of this application; Figure 3(c) is a sagittal plane image of radar characterization of the arm extension posture of a simulated human body in SMPL provided in an embodiment of this application; Figure 4(a) is the 11th frame simulation of the human walking posture generated based on the motion diffusion model provided in the embodiment of this application; Figure 4(b) is the 17th frame simulation of the human walking posture generated based on the motion diffusion model provided in the embodiment of this application; Figure 4(c) is the 36th frame simulation of the human walking posture generated based on the motion diffusion model provided in the embodiment of this application; Figure 4(d) is the 44th frame simulation image of the human walking posture generated based on the motion diffusion model provided in the embodiment of this application; Figure 4(e) is the 66th frame simulation of the human walking posture generated based on the motion diffusion model provided in the embodiment of this application; Figure 5This is a micro-Doppler frequency map of radar characterization of SMPL simulated human walking provided in an embodiment of this application; Figure 6 This is a schematic diagram of a text-command-driven human motion radar scattering simulation device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] In radar-based human behavior perception research, existing human radar scattering simulation methods are limited by the fixed nature of preset motion templates, making it difficult to generate semantically constrained actions based on text commands. Specifically, these methods rely on manually constructed trajectories or limited rule-based action libraries, resulting in a lack of flexibility in the types of actions generated. They cannot generate human motion processes with complex semantic constraints as needed, leading to insufficient diversity in simulation data and impacting the efficiency of constructing radar human datasets and the generalization ability of subsequent algorithm training.

[0014] If the aforementioned problems are not addressed, the construction of radar human body datasets will continue to rely on costly hardware acquisition and time-consuming manual annotation processes, making it difficult to guarantee data consistency and meet the training requirements of large-scale deep learning models. Consequently, the robustness and adaptability of radar human behavior recognition systems will be limited when dealing with complex factors such as environmental noise, individual differences, and occlusion, hindering the effective application of related technologies in real-world scenarios.

[0015] To solve the above technical problems, such as Figure 1 As shown in one embodiment of this application, a text-command-driven human motion radar scattering simulation method is provided, the method comprising: Step S110 involves fusing the semantic feature vector extracted from the text instruction with a preset action prior vector to obtain a semantic condition vector; the text instruction includes action description information.

[0016] Step S120: Generate a human skeleton action sequence based on the semantic condition vector; the human skeleton action sequence includes the three-dimensional spatial position and rotation parameters of the human joints in each time frame.

[0017] Step S130: Construct a three-dimensional human body mesh sequence based on the human skeletal movement sequence.

[0018] Step S140: Based on radar parameters and the three-dimensional human body mesh sequence, the radar echo data cube is calculated.

[0019] Step S150: Based on the radar echo data cube, complete the multi-dimensional imaging visualization.

[0020] For ease of understanding, the following explains some key terms in this embodiment: Text instructions are textual information expressed in natural language that describes the desired human actions.

[0021] Semantic feature vectors are numerical representations extracted from text instructions using natural language processing techniques.

[0022] Action prior vectors are pre-defined numerical vectors that represent common or basic human movement patterns.

[0023] The semantic condition vector is a comprehensive vector obtained by fusing the semantic feature vector and the action prior vector.

[0024] Human skeletal motion sequence is a series of data frames that describe the movement state of the human skeleton over a period of time.

[0025] A human body 3D mesh sequence is a data structure that represents the 3D geometric shape of the human body at consecutive points in time.

[0026] Radar parameters refer to various indicators used to describe the operating characteristics of a radar system, such as transmission frequency, bandwidth, pulse repetition frequency, antenna array configuration, observation range, and angle.

[0027] A radar echo data cube is a collection of time-domain or frequency-domain data received and processed by a radar system, containing target scattering information.

[0028] Multidimensional imaging visualization is the process of converting information from a cube of radar echo data into a three-dimensional image or animation that can be observed by the human eye. Multidimensional imaging visualization can be either micro-Doppler feature maps (two-dimensional visualization) or three-dimensional visualization. The visualization results can intuitively show the position, shape, and motion of a target in multidimensional space.

[0029] This embodiment provides a text-instruction-driven human motion radar scattering simulation method. In step S110, the semantic feature vector extracted from the text instruction is first fused with a preset action prior vector to obtain a semantic condition vector. In practice, the extraction of semantic feature vectors can be achieved in various ways. For example, a method based on the Bag-of-Words model or TF-IDF (Term Frequency-Inverse Document Frequency) can be used to map the keywords of the action description information in the text instruction into numerical vectors.

[0030] Prior vectors for actions can be set by expert experience or obtained through statistical analysis and clustering of a large amount of existing action data. For example, prior vectors representing basic actions such as standing, walking, and running can be preset.

[0031] The fusion of semantic feature vectors and action prior vectors can be achieved through vector concatenation, weighted summation, or more complex neural network layers to ensure the effective combination of semantic constraints and general action knowledge.

[0032] Step S120 is the key to realizing the conversion of text instructions into human actions. It can be implemented by building a rule-based system. The system selects or combines skeletal action segments from a predefined action library based on specific keywords or numerical ranges in the semantic condition vector. For example, when the semantic condition vector indicates walking, the system can retrieve and splice a series of skeletal frames of walking gait from the library.

[0033] One way to implement step S130 is to predefine a simplified geometric model, such as a stick figure model composed of cylinders and spheres, and adjust the size and posture of these geometries according to the position and orientation of the skeletal joints. Another approach is to use a template-based method, that is, to select a basic 3D human body model template and, based on the joint transformation information in the skeletal motion sequence, map the skeletal motion onto the vertices of the human body model using techniques such as linear blending skinning, thereby driving the deformation of the human body mesh.

[0034] Step S140 simulates the interaction between radar signals and a human target. The three-dimensional mesh sequence of the human body can be simplified into a series of point scatterers, each representing a tiny area on the human surface. Then, based on radar parameters and the position and velocity of each point scatterer, the echo signal of each point scatterer is calculated using simple radar equations, and the echoes of all point scatterers are superimposed to obtain the total echo.

[0035] Step S150 aims to transform abstract radar echo data into intuitive multidimensional images (including two-dimensional and three-dimensional images). One approach is to perform simple thresholding and volume rendering on the radar echo data cube, directly mapping data points above a certain threshold to voxels in three-dimensional space and assigning them different colors or transparency for display. Another approach is to perform a Fourier transform on the radar echo data, converting it from the time domain to the frequency domain, extracting range and Doppler information, and then projecting this information onto a two-dimensional plane to generate a range-Doppler map or a micro-Doppler frequency map (two-dimensional map). Three-dimensional information can then be indirectly represented by stacking these two-dimensional maps. These methods may have limitations in generating high-resolution, highly realistic multidimensional imaging results; for example, it may be difficult to accurately reconstruct the fine structure of a target or distinguish closely adjacent targets.

[0036] The following example will provide a more detailed explanation of the above technical solution: Suppose user A wants to simulate the motion of a human jogging on a treadmill and obtain its radar scattering data.

[0037] First, user A inputs the text command to jog on a treadmill. After the text command is received, a semantic feature vector is extracted from it. For example, a pre-trained language model converts words like "treadmill" and "jogging" into numerical representations, and combines this with contextual information to form a comprehensive semantic feature vector. Simultaneously, pre-defined action prior vectors, such as vectors representing common action patterns like walking and running, are fused with the semantic feature vector to obtain a semantic condition vector. This semantic condition vector precisely encodes the semantic constraints of the specific action of jogging on a treadmill.

[0038] Next, a sequence of human skeletal movements is generated based on the semantic condition vector. For example, a deep learning-based generative model receives the semantic condition vector as input and outputs a series of continuous skeletal joint position and rotation data. These data frames collectively describe the movement trajectory of key joints such as the hip, knee, ankle, shoulder, and elbow as a person jogs on a treadmill over time. The skeletal movement sequence not only includes the overall rhythm of the movement but also reflects the gait characteristics unique to jogging.

[0039] Subsequently, a sequence of 3D human body meshes is constructed based on the sequence of human skeletal movements. For example, the generated skeletal movement sequence is input into the skinning function of a parametric human model (such as an SMPL model) and combined with preset body shape parameters. The skinning function drives the vertex deformation of the human body surface mesh in real time according to the movement of the skeleton, thereby generating a series of continuous 3D human body mesh sequences. These mesh models accurately simulate details such as body posture, muscle deformation, and clothing wrinkles during jogging.

[0040] Based on this, a radar echo data cube is calculated using preset radar parameters and a 3D human body mesh sequence. Specifically, the generated 3D human body mesh sequence is traversed frame by frame. For each frame, the mesh is divided into smaller triangular elements, and the complex scattering field of each triangular element to the radar signal is calculated based on parameters such as radar transmission frequency, bandwidth, pulse repetition frequency, antenna position, and the position, orientation, and material properties of each triangular element. Subsequently, the complex scattering fields of all triangular elements are summed to obtain the noise-free frequency domain echo of the frame. To simulate a real environment, complex Gaussian white noise is added to the noise-free echo to form a noisy frequency domain echo. By repeating this process for all time frames, a radar echo data cube containing range, Doppler, and time dimension information is finally generated.

[0041] Finally, multi-dimensional imaging visualization is achieved based on the radar echo data cube. For example, pulse compression processing is performed on the radar echo data cube to transform it into the range domain, resulting in a series of range images. Then, based on these range images, micro-Doppler frequency maps and range-Doppler maps can be generated to analyze the fine features of human motion. Simultaneously, by combining the virtual aperture characteristics of the MIMO array, the range images are processed through a back projection algorithm, ultimately forming an intuitive multi-dimensional imaging result. The multi-dimensional imaging result can clearly display the human body contour, movement trajectory, and relative motion of various body parts of user A while jogging on a treadmill, thus realizing text-command-driven radar scattering simulation of human motion.

[0042] This method first fuses semantic feature vectors extracted from text with prior action vectors to obtain semantic condition vectors. Based on these vectors, a sequence of human skeletal movements is automatically generated. Then, combined with body shape parameters, a 3D human mesh sequence is constructed. Electromagnetic scattering calculations are performed based on radar parameters and the mesh model to obtain a radar echo data cube. Finally, multi-dimensional visualization imaging is achieved through signal processing. This method completely eliminates the limitations of traditional radar data acquisition, which relies on expensive hardware, dedicated facilities, and significant manpower, greatly reducing data acquisition costs and time. It can quickly produce standardized simulation data in batches. Simultaneously, it overcomes the drawback of traditional simulations that can only use fixed action templates. Relying on text semantics combined with prior constraints, it flexibly generates various complex human movements, significantly improving semantic controllability and movement diversity. The entire chain achieves fully automated integrated operation from natural language input to radar imaging, without the need for manual intervention in designing action trajectories and simulation parameters. The generated human movements closely resemble real motion patterns, and the 3D triangular mesh geometry is reasonable. The radar echoes obtained by combining physical optics methods and complex Gaussian noise simulation highly reproduce the real detection environment. The final output multi-dimensional imaging has outstanding practicality and engineering application value.

[0043] In some embodiments of this application, a human skeletal action sequence is generated based on a semantic condition vector, including: Generate a sequence of potential actions based on the semantic condition vector; Generate a sequence of motion parameters based on the potential action sequence and semantic condition vector; Generate a sequence of human skeletal movements based on the sequence of motion parameters.

[0044] Among them, latent action sequences are intermediate representations, low-dimensional, abstract sequences that capture the core dynamic and structural information of human movements. The purpose of generating latent action sequences is to decompose the complex action generation problem into more manageable and learnable stages. These can be latent space representation sequences obtained by progressively denoising noise from noise using a diffusion model.

[0045] Motion parameter sequences are a more specific representation of motion, describing the posture information of various joints in the human body as they change over time, such as the rotation angle or relative position of each joint. The sequence is obtained by further refining the latent motion sequence and semantic condition vector, and contains detailed joint motion data required to generate human skeletal movements. It can be, for example, a sequence of joint angles output by decoding the latent motion sequence by a neural network model and combining it with the semantic condition vector, or a sequence obtained by converting key point positions into joint angles through an inverse kinematics solver.

[0046] The human skeleton motion sequence is the final motion representation. It contains complete pose information of the human skeleton model in continuous time steps, typically including the global position, orientation, and relative rotation of each bone and joint. The sequence is built based on the motion parameter sequence and can be directly used to drive the 3D human model for animation or simulation. It can be obtained, for example, by applying the joint angles in the motion parameter sequence to a predefined human skeleton topology, or by converting the joint angles into a sequence of skeleton transformation matrices.

[0047] This method decomposes the process of generating human skeleton motion sequences from semantic condition vectors into a multi-stage generation mechanism, making the conversion from abstract text instructions to specific human skeleton movements more stable and controllable. First, a latent motion sequence is generated based on the semantic condition vector. This sequence acts as a bridge between semantic information and specific motion details, capturing the high-level dynamic features of the motion and reducing the difficulty of directly jumping from abstract semantics to high-dimensional skeleton details. Subsequently, the latent motion sequence and the original semantic condition vector are further utilized to generate a more specific motion parameter sequence, ensuring that the joint motion sequence decoded from the latent space still conforms to the expected semantic instructions. At the same time, the latent motion sequence provides temporal coherence and dynamism of the motion. Finally, based on the generated motion parameter sequence, a complete human skeleton motion sequence is constructed. This staged and progressively refined generation method effectively avoids the problems of unnatural or inaccurate motions that may result from direct generation. It ensures that the generated human skeleton motion sequence is not only highly consistent with the text instructions semantically, but also exhibits smooth and realistic motion characteristics physically, thus providing high-quality input for subsequent radar scattering simulations.

[0048] In some embodiments of this application, generating a potential action sequence based on a semantic condition vector includes: Using semantic condition vectors as semantic constraints and standard Gaussian noise sequences as diffusion starting points, DDIM sampling is performed step by step; After sampling is completed in all diffusion steps, the potential action sequence is obtained.

[0049] Semantic constraints refer to using semantic condition vectors as guiding information during the generation of potential action sequences to ensure that the generated results are consistent with the semantic content expressed by the text instructions.

[0050] A standard Gaussian noise sequence is a random sequence that follows a standard normal distribution (mean 0, variance 1). In action diffusion models, it is usually used as the initial input to the generation process, representing a highly disordered and random state. By gradually denoising from this random state, the model can explore a broad potential space, thereby generating diverse potential action sequences.

[0051] DDIM (Denoising Diffusion Implicit Models) sampling is an efficient and deterministic motion diffusion model sampling method. It iteratively denoises from noise to gradually recover data samples. In each sampling step, DDIM uses the learned denoising model (a sub-model in the motion diffusion model) to predict the data distribution under the current noise state and updates the samples based on the prediction, thereby generating high-quality samples in fewer steps. The latent motion sequence is a low-dimensional or abstract representation of a specific human action that is finally obtained through the DDIM sampling process. It is usually a temporal sequence in which each element encodes the key features of the action at a certain moment, such as joint angles, displacements, or posture information. This sequence is the basis for the subsequent generation of specific motion parameter sequences.

[0052] This method uses semantic condition vectors as semantic constraints in the generation process and standard Gaussian noise sequences as the starting point for diffusion. It utilizes the DDIM sampling mechanism to progressively denoise, thereby generating latent action sequences. Specifically, in each iteration of DDIM sampling, the denoising model receives the noise sequence of the current state and the semantic condition vector as input. The semantic condition vector plays a guiding role in this process, ensuring that the denoising model integrates semantic information into the generation of latent action sequences while removing noise. Through this iterative denoising and semantic guidance, the model can progressively extract structured latent action sequences that are highly correlated with the semantic condition vector from completely random noise. This mechanism effectively solves the problems of semantic inconsistency and unnatural movements that may occur when directly generating complex action sequences from semantic condition vectors, laying a high-quality foundation for the subsequent generation of realistic human skeletal action sequences.

[0053] In some embodiments of this application, a sequence of motion parameters is generated based on a potential action sequence and a semantic condition vector, including: The potential action sequence and semantic condition vector are input into the temporal downsampling variational autocoder to obtain the normalized motion parameter sequence output by the temporal downsampling variational autocoder. The normalized motion parameter sequence is denormalized to obtain the motion parameter sequence; Based on the sequence of motion parameters, a sequence of human skeletal movements is generated, including: The global rotation, local rotation of body joints, and translation of root node are extracted from the motion parameter sequence for each frame. Convert global rotation and local rotation of body joints into global axis angle and body joint axis angle, respectively; By integrating the global axis angle, body joint axis angle, and root node translation of each frame, a human skeletal motion sequence is obtained.

[0054] Temporal downsampling variational autoencoder is a neural network structure capable of processing temporal data and performing dimensionality reduction or expansion transformations. Its core lies in its ability to decode high-dimensional temporal data from the latent space. The decoding end is responsible for mapping low-dimensional latent representations and conditional information to a high-dimensional output space. It can adopt various architectures, such as recurrent neural networks or Transformer architectures, combined with downsampling operations to handle features at different time scales. Normalized motion parameter sequences refer to the process of transforming various motion parameters of the original human joint movements through specific mathematical transformations so that their numerical range is within a preset interval. The purpose of normalization is to eliminate the dimensional differences between different parameters, improve the stability and convergence speed of model training, and prevent certain parameters from having an excessive impact on the model output. Inverse normalization is the reverse process of normalization, that is, restoring the normalized motion parameter sequence to its original numerical range and physical meaning. This step is necessary because the model is usually trained and inferred on normalized data, and the final output motion sequence needs to be presented in the form of actual physical quantities for subsequent rendering or simulation.

[0055] This method first takes the latent action sequence and semantic condition vector as input and feeds them into a temporal downsampling variational autoencoder. Leveraging its temporal processing capabilities and decoding mechanism, the autoencoder effectively fuses low-dimensional latent action information with high-dimensional semantic condition information, mapping them to a high-dimensional motion parameter space. This outputs a normalized motion parameter sequence. This design allows the model to fully utilize semantic information to guide action generation, ensuring that the generated actions conform to the semantic requirements of the text instructions. Subsequently, the normalized motion parameter sequence is denormalized to restore it to its actual physical dimensions, thus obtaining a specific motion parameter sequence. This sequence contains detailed motion information of each joint of the human body at different time steps. Based on this, to construct a complete human skeletal action sequence from the motion parameter sequence, global axis angles, joint axis angles, and root node displacements are further extracted from the motion parameter sequence of each frame. Global axis angles define the overall orientation of the body, joint axis angles describe the relative rotation of each joint, and root node displacements characterize the body's movement in space. By integrating these key motion parameters, the human skeleton pose of each frame can be accurately reconstructed, ultimately forming a continuous sequence of human skeleton movements. This step-by-step refinement and integration strategy effectively solves the problem of transforming abstract action representations into concrete skeletal movements, ensuring the semantic consistency, physical rationality, and temporal coherence of the generated actions, and providing high-fidelity human motion input for subsequent radar scattering simulations.

[0056] In some embodiments of this application, a three-dimensional human body mesh sequence is constructed based on the human skeletal movement sequence, including: The human skeleton motion sequence and body shape parameters are input into the SMPL skinning function to obtain the human body 3D mesh sequence output by the SMPL skinning function.

[0057] Among them, the human skeleton motion sequence is a series of data describing the posture and position of the human skeleton at different time points. It can consist of a series of joint rotation and translation vectors, each vector corresponding to a time step, or it can consist of a sequence of coordinate points of the skeletal joints in three-dimensional space. Its function is to serve as the basic input to drive the deformation of the three-dimensional human body model.

[0058] Body shape parameters define the basic shape and size of the human body, such as height, weight, and body type. These can be low-dimensional vectors describing the principal components of the human body shape, such as the shape parameters (beta parameters) in the SMPL model, or specific dimensional data obtained through scan data or user input, such as chest circumference, waist circumference, and hip circumference. They are crucial for constructing personalized 3D human body models.

[0059] The SMPL skinning function is a function that maps shape and pose parameters to the vertex positions of a human body mesh. Based on the principle of Linear Blend Skinning (LBS), it applies skeletal transformations to mesh vertices through predefined skinning weights. Its function is to transform abstract skeletal motion and body shape information into a concrete, geometrically detailed 3D human body model.

[0060] A human body 3D mesh sequence is a sequence of time-continuous 3D triangular meshes. Each mesh represents the geometry of the human body at a certain moment. It can be composed of a series of standard 3D model files such as OBJ and FBX, with each file corresponding to a time frame, or represented by a data structure in memory, containing the vertex coordinates, normal vectors, and face indices of each time frame. Its function is to serve as the geometric input for radar scattering simulation, providing detailed information about the human body surface.

[0061] This method provides the human skeleton motion sequence with posture information of the human body at different times, while body shape parameters define the inherent shape of the human body. The SMPL skinning function, as the core processing unit, organically combines this posture and shape information. It drives the deformation of mesh vertices according to the motion of the skeleton and adjusts the overall shape of the mesh according to the body shape parameters, thereby generating a series of temporally continuous 3D human body mesh sequences. This method ensures that the generated human body model is not only accurate in posture but also realistic in body shape, providing high-fidelity geometric input for subsequent radar scattering simulation. In this way, this method effectively solves the challenge of transforming abstract skeletal motion into a realistic 3D human body model, significantly improving the accuracy and reliability of simulation results.

[0062] In some embodiments of this application, a radar echo data cube is calculated based on radar parameters and a three-dimensional human body mesh sequence, including: The system iterates through each frame of the human body 3D mesh sequence frame by frame, and obtains the triangular element parameters corresponding to each frame mesh by performing triangular element subdivision on each frame mesh. Based on the parameters of the triangular surface element and the radar parameters, the complex scattering field of the triangular surface element is obtained; By summing the complex scattering fields of all the triangular facets, a noiseless frequency domain echo of a single frame is obtained. Adding complex Gaussian white noise to the noiseless frequency domain echo of a single frame yields a noisy frequency domain echo of the same frame. A radar echo data cube is generated based on the noisy frequency domain echo of a single frame.

[0063] The process involves traversing each frame of the human body 3D mesh sequence frame by frame and performing triangular subdivision, aiming to decompose the complex 3D human body model into smaller, computable basic scattering units.

[0064] Triangular meshing decomposes each mesh into a series of tiny planar triangles. These triangles are the basic units for calculating radar scattering. For example, standard 3D modeling libraries such as Open3D or VTK can be used to parse the input 3D human body mesh sequence, extract its vertex, edge, and face information, and treat each face as a triangular mesh element. Based on the parameters of the triangular facet element and the radar parameters, the complex scattering field of the triangular facet element is obtained. This step is to calculate the contribution of each tiny scattering unit to the radar signal. The complex scattering field contains the amplitude (intensity) and phase information of the scattered signal and is the basis of the radar echo. For example, physical optics (PO) can be used to calculate the scattering coefficient and phase delay of each triangular facet element by combining its geometric parameters (such as normal vector, area, and position) and radar parameters (such as frequency, incident angle, and polarization), thereby obtaining the complex scattering field.

[0065] The complex summation of the complex scattered fields from all triangular elements yields the noise-free frequency domain echo for a single frame. This step involves superimposing the contributions of all the tiny scattering units to form the overall radar echo at a given moment (single frame). The complex summation takes into account the phase differences between different scattering points, simulating the interference effect of radar signals. For example, the complex scattered fields calculated from all triangular elements can be directly vector-superimposed, i.e., the real and imaginary parts of each complex scattered field are summed separately to obtain a total complex number representing the noise-free frequency domain echo of the frame.

[0066] Adding complex Gaussian white noise to the noiseless frequency domain echo to obtain a noisy frequency domain echo of a single frame is a step to make the simulation results closer to the real-world radar echo, because various noises inevitably exist in actual radar systems, among which complex Gaussian white noise is the main internal noise source.

[0067] Generating a radar echo data cube from each frame of noisy frequency-domain echoes is a step that organizes the noisy frequency-domain echoes from all time frames into a unified data structure, facilitating subsequent radar signal processing and visualization. The data cube is typically represented as three-dimensional data in the time-frequency-range (or time-frequency-Doppler) framework. For example, the noisy frequency-domain echoes calculated for each frame can be stacked in chronological order to form a two-dimensional matrix (time-frequency). If the radar system is multi-channel, it can be expanded to three dimensions (time-frequency-channel), thus forming the radar echo data cube.

[0068] This method first processes the three-dimensional human body mesh sequence frame by frame when calculating the radar echo data cube. The complex human geometry of each frame is decomposed into a series of basic triangular facets. This transforms the continuously changing dynamic human body model into a discrete, computable set of scattering units. Then, for each triangular facet, combined with preset radar parameters, its complex scattering field to the radar signal is precisely calculated. This includes the amplitude and phase information of the scattered signal, which is crucial for simulating the interaction between radar waves and the human body surface. Next, the complex scattering fields of all triangular facets within the same frame are summed to comprehensively superimpose the contributions of each scattering point, thus obtaining the noise-free frequency domain echo at each moment. To make the simulation results closer to real-world scenarios, this method further introduces complex Gaussian white noise into the noise-free frequency domain echo to simulate the unavoidable internal noise in actual radar receivers, generating noisy frequency domain echoes. Finally, the noisy frequency domain echoes from different time frames are organized in a time sequence to form the radar echo data cube. Through the above steps, this method can systematically and physically simulate the echo characteristics of human body movements in a radar system, providing a real and reliable data foundation for subsequent multidimensional imaging visualization, and effectively solving the problem of how to accurately simulate the interaction between radar waves and human body grids and introduce real noise.

[0069] In some embodiments of this application, multi-dimensional imaging visualization is performed based on the radar echo data cube, including: The radar echo data cube is pulse-compressed and converted to the range domain to obtain a range image; Based on the range image, micro-Doppler frequency maps, range-Doppler maps, and range-time maps are generated respectively. Then, based on the range image and the MIMO array, a multi-dimensional imaging visualization is formed through a back projection algorithm.

[0070] The process of pulse compression processing on radar echo data cubes and converting them to the range domain to obtain a range image refers to improving the radar's range resolution and signal-to-noise ratio through radar signal processing technology, and mapping the radar echo data to the range dimension to display the echo intensity distribution of the target at different distances. The processing can be performed on the radar echo data cubes using a matched filter. The matched filter can maximize the output signal-to-noise ratio and achieve pulse compression. Subsequently, an inverse Fourier transform is performed on the matched-filtered data to obtain the range image in the range domain.

[0071] Generating micro-Doppler frequency maps and range-Doppler maps based on range images refers to using two-dimensional visualization methods commonly used in radar signal processing to reveal the subtle Doppler frequency shifts caused by the internal motion of a target, and simultaneously displaying the target's range and velocity information. When generating micro-Doppler frequency maps, short-time Fourier transform (STFT) or wavelet transform can be performed on the echo signals of specific range cells in the range image to analyze their time-frequency characteristics. When generating range-Doppler maps, Fourier transform is typically performed on each range cell in the range image to obtain the Doppler spectrum of the range cell. Then, the Doppler spectra of all range cells are combined to form a two-dimensional map. Alternatively, more advanced time-frequency analysis methods such as the Wigner-Ville distribution and the Ambiguity function can be used to generate micro-Doppler frequency maps to achieve higher time-frequency resolution. For range-Doppler maps, fast Fourier transform (FFT) can be used to process the echo data of each range gate to efficiently extract Doppler information.

[0072] Multidimensional imaging visualization is formed by combining range images with MIMO arrays and using back-projection algorithms. This refers to leveraging the advantages of MIMO array radar, forming a virtual array through multiple transmitting and receiving antennas to improve angular resolution and spatial sampling capabilities, and then using the back-projection algorithm (BPA) to reconstruct two-dimensional or three-dimensional images of the target.

[0073] By combining MIMO arrays, the characteristics of their virtual arrays can be utilized to process range image data received from different virtual antenna pairs. The specific implementation of the back projection algorithm can include: first, defining a multi-dimensional imaging region and discretizing it into a series of voxel points; then, for each voxel point, calculating its distance and angle to each MIMO virtual antenna pair; next, superimposing the echo intensity values ​​corresponding to the distance and angle in the range image onto the voxel point after phase compensation; and finally, traversing all voxel points and all virtual antenna pairs to form the final multi-dimensional imaging result.

[0074] This method combines the advantages of MIMO array radar. MIMO arrays, through their multiple transmit / receive configuration, effectively expand the virtual aperture, significantly improving the angular resolution of the radar system. Building upon this, a back projection algorithm is employed to project the range image data, after pulse compression and range domain transformation, along with the spatial information from the MIMO array, into three-dimensional space. The back projection algorithm coherently superimposes echo signals from different antennas, thereby accurately reconstructing the geometry and trajectory of the human target in three-dimensional space. Through this multi-dimensional data processing and advanced imaging algorithms, this method can efficiently and accurately extract spatial, range, velocity, and micro-motion information of human movement from complex radar echo data. This solves the problem of the difficulty in directly visualizing and analyzing raw radar echo data, providing high-quality input for subsequent applications such as human motion recognition and behavior analysis.

[0075] like Figures 2 to 5 As shown, for ease of understanding, some embodiments of this application provide the following examples: Step S210: Receive text instructions input by the user and extract semantic condition vectors.

[0076] Text instructions include action description information, specifically natural language text or action category labels; If it is an action category label, a pre-trained text encoder is used to extract its semantic features and generate a semantic representation of the target action.

[0077] Let the set of action categories be: (1); in, Indicates the first Action categories, This represents the total number of action categories.

[0078] For the input action description information It can be converted into a semantic representation of the target action through predefined vocabulary mapping, rule matching, or semantic encoding. , denoted as: (2); in, This represents the action semantic parsing function.

[0079] For natural language text, semantic feature vectors are extracted using a pre-trained text encoder and a conditional projection network: (3); in, This represents a pre-trained text encoder. This represents a semantic feature vector.

[0080] semantic feature vector The vector is obtained by mapping to the conditional space through a conditional projection network. : (4); in, This represents a conditional projection network, which can be implemented using a linear layer or a multilayer perceptron.

[0081] Next, or By fusing with the action prior vector, a unified action condition representation is obtained, which is the semantic condition vector: (5); in, This represents the conditional fusion function. This represents the action prior vector constructed from action category priors. Semantic condition vector This serves as the input condition for the subsequent action diffusion model.

[0082] Step S220: Construct an action diffusion model (based on a temporal downsampling variational self-encoder, and it should be noted that only the decoding end is used in the inference stage, and the encoding end only participates in offline pre-training), and then train it.

[0083] To reduce the computational overhead of high-dimensional action sequences on the action diffusion model while maintaining temporal consistency of generated actions, this method first constructs and trains an action diffusion model. A temporal compression structure is introduced at the encoding end to compress the motion parameter sequence while preserving key dynamic features of the motion. Mapping to shorter, lower-dimensional potential action sequences The decoding end is responsible for reconstructing the complete action from the latent action sequence. After pre-training, the parameters of the action diffusion model are frozen, and all subsequent training and inference of the action diffusion model are carried out in the low-dimensional latent space, which effectively improves computational efficiency and preserves the temporal smoothness characteristics.

[0084] Within the potential space, with For semantic guidance, the forward diffusion process is defined as gradually moving towards the potential action sequence according to a preset noise schedule. Superimposed Gaussian noise: (6); Accordingly, denoising model Responsible for a given number of diffusion steps and The model predicts the current noise component under certain conditions. It employs a temporal self-attention structure and injects both the temporal embedding of the diffusion steps and the action conditional embedding into the sequence representation, thereby effectively guiding the denoising process with semantic information. To improve the physical plausibility and temporal smoothness of the synthesized actions, a composite training objective is designed.

[0085] make This represents the noise in the prediction, from which the true potential action sequence (or a clean potential action sequence) can be deduced. Its comprehensive loss function consists of three terms: Noise prediction loss This is used to constrain the denoising model to accurately recover the injected noise; Reconstruction losses This allows the potential action sequence derived from the noise to be deduced. With the actual potential action sequence Maintain consistency; Speed ​​consistency loss Constraining the potential action sequences of adjacent frames The changing patterns of these factors contribute to a smoother sequence of generated actions. The final training loss is a weighted combination of the above factors: (7); in and These are the preset weight coefficients. By minimizing this loss, the model fully learns the potential distribution patterns of different actions under the constraints of the category conditions, ensuring that the generated results are both semantically accurate and have natural motion.

[0086] After training, for any target action category, the initial value is random Gaussian noise. Starting from this point, noise is gradually reduced through back-diffusion iteration to obtain the true latent action sequence under the given constraints. Then, a semantically correct, smooth, and natural sequence of human skeletal movements is reconstructed using a pre-trained autoencoder decoder. .

[0087] Step S230, Input the trained motion diffusion model and generate a sequence of human skeletal motions through forward inference.

[0088] semantic condition vector The trained motion diffusion model is input, and forward inference generates a human skeleton motion sequence that is consistent with the semantics, including the local rotation parameters of each joint of the human body and the global displacement of the root node at each time frame.

[0089] The reasoning process is completed in two phases: the latent space and the motion space. First, in the latent space, using... or The corresponding action condition is embedded as a guide, from standard Gaussian noise. Starting with a denoising model Inverse diffusion denoising is performed. To ensure a uniquely determined generation result for the same semantic condition and to improve inference efficiency, this method adopts a deterministic DDIM sampling strategy: (8); in The true potential action sequence estimated for the current step .

[0090] The potential action sequence is obtained after iteration. ,in, This represents the potential number of time steps after compression. As a latent dimension, the latent action sequence Together with the action condition embedding, the data is sent to the decoding end of the action diffusion model. Decode it into a normalized sequence of motion parameters : (9); After inverse normalization to restore the absolute motion parameter space, the motion parameter sequence is obtained. : (10); Motion parameter sequence Each frame can be broken down into three components: global rotation, local rotation of body joints, and translation of the root node.

[0091] In this model, both global rotation and rotation of each joint of the body are represented using a continuous 6-dimensional rotation to avoid ambiguity and discontinuity that may occur with quaternion or axis-angle representations in neural network regression. The 6-dimensional rotation representation is then converted into axis-angle form to obtain skeletal movements that can directly drive the human parametric model. The rotation matrix is ​​recovered from the 6-dimensional vector through an orthogonalization process, and then the rotation matrix is ​​transformed into global axis-angle representations using the matrix-axis-angle transformation relationship. and the joint axis angle of each body .

[0092] The final generated human skeletal motion sequence can be represented as: .

[0093] Step S240: Use SMPL to parametrically model the human body, based on the human skeletal motion sequence. The parameters of each frame serve as the driving input. The coordinates of the vertices of the human body 3D mesh in each frame are calculated using a linear hybrid skinning algorithm to obtain the human body 3D mesh sequence.

[0094] The generated human skeletal motion sequence As the driving input, the SMPL parametric human body model is used to reconstruct a 3D human body mesh sequence.

[0095] For the Human skeleton motion sequence of frames First, the 6-dimensional continuous rotation representation is converted into the axis-angle representation required by the SMPL model. Global rotation Local rotation of various body joints The rotation matrix is ​​reconstructed through orthogonalization and then converted into axis-angle vectors using the Rodriguez formula. Root node translation. It is obtained directly from the accumulated relative translation.

[0096] Let the human body shape parameters be... This, along with global rotation, body joint rotation, and root node translation, is fed into the skinning function of the SMPL model: (11); in, For the first The set of vertices of a 3D mesh sequence of the human body in frames. This represents the 3D coordinates of all joints. Internally, the skinning function uses shape blending, pose deformation, and linear blending skinning algorithms to propagate the rigid body transformations of the joints driven by skeletal motion to all mesh vertices, generating human surface deformations consistent with joint movement. For all... The above process is repeated for each frame, eventually generating a 3D human body mesh sequence. And the corresponding joint trajectories. The sequence fully preserves the motion semantics output by the motion generation model, while ensuring the rationality of the human body geometry through the physiological constraints of the SMPL model.

[0097] Step S250: Perform electromagnetic scattering modeling and radar forward simulation on the three-dimensional mesh sequence of the human body to obtain the radar echo data cube.

[0098] Based on the set radar parameters, a triangular surface element scattering model based on physical optics (PO) is used to calculate the scattering contribution of each visible surface element in each frame of the human body mesh, and the radar echo of each frame is obtained by coherent superposition.

[0099] Discretize each frame of the human body mesh using surface elements: Let the first The frame human body mesh is composed of It consists of triangular facets, each facet defined by the coordinates of its three vertices. For the ... Given a triangular element, calculate its center position. Surface element normal vector and surface area .

[0100] The radar employs an ultra-wideband stepped-frequency continuous wave system and is equipped with... One transmitting antenna and There are 1 receiving antenna, forming a total of 100 receiving antennas. A dual-station channel. Assume the first... The positions of the transmitting antennas for each channel are as follows: The receiving antenna is located at The two-way propagation path length of the surface element under this channel is: (12); Based on the high-frequency approximation of physical optics, a bistatic scattering model of surface elements is established, where the unit vector of the emission direction is assumed to be... The receiving direction unit vector is The local incident angle of each surface element is determined based on the geometric relationship between the surface element normal and the transmission / reception direction. With local reflection angle The scattering contribution of a single surface element is determined by both the amplitude factor and the phase factor. The amplitude factor reflects the influence of the surface element area, viewing angle, and frequency on the scattering intensity. (13); in For frequency The corresponding wavelength, The speed of light. The phase factor is determined by the phase delay caused by the two-way propagation path length: (14); The received signal strength attenuates according to the spherical wave spread, with an attenuation factor of . Considering the combined amplitude, phase, and distance attenuation, therefore the first... Each element in the first Channels, frequencies The contribution of the scattered field at that location is: (15); Traverse all valid surface elements in the current frame, and coherently superimpose the scattering contributions of the same channel to obtain the th element. Frame, First Noise-free frequency domain echoes for each channel: (16); To simulate the complex white Gaussian noise environment in a real radar receiver, additive white Gaussian noise is injected into the synthesized echo. Let the noiseless echo power be... Given a preset signal-to-noise ratio The noise power meets the requirements. The noisy frequency domain echo after adding complex Gaussian noise is: (17); Radar in Equally spaced frequency points Collect data. Repeat the above process for all transmit and receive channels and frequency points to obtain the first... The multi-channel scattering matrix of the frame, traversing all The frames ultimately generate a complete multi-frame, multi-channel complex frequency domain radar echo data cube. .

[0101] Step S260: Perform signal processing and imaging characterization on the radar echo data cube, including obtaining a micro-Doppler frequency map by short-time Fourier transform, obtaining a range-time map by pulse compression, obtaining a range-Doppler map by Doppler transform, and obtaining multi-dimensional imaging results by back projection algorithm.

[0102] The generated multi-frame, multi-channel complex frequency domain echo data cube is processed to characterize the radar scattering features of human motion from both time and frequency domains and spatial domains, resulting in a two-dimensional micro-Doppler spectrum and a three-dimensional back projection imaging result.

[0103] First, pulse compression is performed along the frequency dimension to convert the frequency domain echo into the range domain.

[0104] For the Frame, First Frequency domain echo of each channel By performing an inverse Fourier transform and applying a window function to suppress range sidelobes, the range image of that channel in that frame is obtained. : (18); in, It is a frequency domain window function, the range axis is determined by radar system parameters, and the range cell size is... To extract Doppler modulation information of human motion, a continuous range gate interval containing the main scattering contribution of the human body is selected in the range profile. The complex signals of each range cell within this interval are summed over slow time to obtain an equivalent slow-time complex sequence. , Its sampling frequency is equal to the system's pulse repetition frequency. .

[0105] For slow time signals By performing a short-time Fourier transform, and by sliding a window function on the time axis and performing Fourier transforms segment by segment, the time-varying spectrum of the signal is obtained: (19); in For time-domain window functions, The time corresponding to the center position of the window. For Doppler frequency, The sampling frequency is slow. The amplitude spectrum of the short-time Fourier transform is normalized and converted to a decibel scale: (20); This yields a micro-Doppler spectrum, with time on the horizontal axis and Doppler frequency on the vertical axis. Grayscale or color coding reflects the strength of each Doppler component at different times. This spectrum visually depicts the instantaneous Doppler frequency shift caused by details of human motion such as limb swings and periodic trunk displacements, clearly distinguishing the characteristic differences of different motion types in the Doppler domain. Furthermore, distance-Doppler and distance-time maps can be further extracted from the short-time Fourier transform results as auxiliary representations.

[0106] Utilizing the multi-channel spatial diversity capability of a MIMO array, a back projection algorithm is used to reconstruct the three-dimensional scattering distribution of a human target in space. A mesh is defined to divide the imaging area into... voxels, each voxel has coordinates as follows: For the first There are one transceiver channel, and the positions of its transmitting and receiving antennas are as follows: , The corresponding distance image is Voxels The round-trip distance to this passage is: (twenty one); Complex scattering values ​​are extracted from the range image based on the two-way distance, and multiplied by a phase compensation factor to eliminate the propagation phase difference between different transmit and receive channels. Coherent superposition is then performed on all channels. (twenty two); in The radar's operating center frequency is used. By iterating through all voxels, a three-dimensional complex image is obtained, and its amplitude is taken. and normalized to The interval is the three-dimensional scattering intensity distribution of the human target.

[0107] The following are experimental examples: To verify the effectiveness of the method at each stage of the entire chain, two sets of experiments, one static and one dynamic, were conducted: By selecting extended arm as a typical static posture and walking as a typical dynamic action, the correctness of the integrated method of text-driven human motion generation and radar scattering simulation is verified. Both experiments cover the core aspects of the method, validating its feasibility.

[0108] The radar simulation system parameters used for verification are as follows: A MIMO radar simulation system with an ultra-wideband stepped-frequency continuous wave (UWB-SFCW) architecture was selected for verification. The radar operates in the frequency band from 1.785 GHz to 2.805 GHz, with a frequency step of 4 MHz, collecting data at 256 frequency points, resulting in an equivalent bandwidth of 1.02 GHz. The system employs a 10-transmit, 10-receive antenna array, forming 100 bistatic virtual channels, with the array center located at [0, -2, 1.5] m. The motion sequences are derived from the publicly available human motion capture dataset AMASS.

[0109] For static verification, the text instruction "A standing person in a T-pose" is input into the algorithm model. Through the same semantic encoding and diffusion generation process, static pose skeletal parameters with arms outstretched laterally are obtained. This drives the SMPL model to reconstruct a 3D human triangular mesh, which fully preserves the upper limb spatial configuration corresponding to the arm-out gesture. Figure 2 As shown.

[0110] Electromagnetic scattering simulation was performed on the human body mesh to obtain multi-channel complex frequency domain echoes. Utilizing the multi-channel spatial diversity capability of the MIMO array, the three-dimensional scattering intensity distribution of the human target was reconstructed using a back projection algorithm. The imaging results of three orthogonal planes were presented using the maximum projection method. Figure 3(a) shows the maximum projection in the horizontal plane (i.e., the XY plane), reflecting the distribution of scattering centers on the horizontal section; Figure 3(b) shows the maximum projection in the coronal plane (i.e., the XZ plane), presenting the contour structure of the human body in the left-right and height directions, clearly identifying the horizontally outstretched arms and torso; Figure 3(c) shows the maximum projection in the sagittal plane (i.e., the YZ plane), depicting the posture details of the human body in the front-back and height directions. The projection results from different perspectives effectively identified the spatial scattering characteristics of the upper torso, shoulders, and arms, verifying the feasibility of the MIMO array back projection algorithm for characterizing the static spatial structure of the human target.

[0111] For dynamic verification, the text command "Walk" or "A man is walking" is obtained and mapped to a semantic condition vector. This vector guides the action diffusion model to generate a human skeletal action sequence consistent with the semantics, which is then reconstructed into a human 3D mesh sequence using the SMPL model. Figures 4(a) to 4(e) show five typical posture diagrams of the walking action generated by the action diffusion model. The diffusion-generated actions are coherent and natural, and the postures of each joint conform to the laws of human kinematics. The gait features such as alternating leg support and coordinated arm swing are obvious. Electromagnetic scattering modeling and simulation based on physical optics is performed frame by frame on the human 3D mesh sequence to obtain multi-frame, multi-channel complex frequency domain echo data cubes. After pulse compression, range gate selection, and short-time Fourier transform processing, micro-Doppler spectra are obtained, such as... Figure 5 As shown. Figure 5The periodic Doppler frequency shift caused by trunk translation and limb swing can be clearly observed. The frequency components change regularly with time, and the gait periodic characteristics are obvious. This is basically consistent with the micro-Doppler theoretical characteristics of real human walking. Thus, the walking motion dynamics prove the integrity and correctness of the link between text-driven dynamic motion generation and radar time-frequency representation.

[0112] The verification results of the selected set of static experiments and set of dynamic experiments show that, starting from text semantics, a motion sequence conforming to the laws of human movement is generated through a conditional diffusion model, reconstructed into a three-dimensional surface mesh using the SMPL model, and then subjected to electromagnetic scattering simulation and signal processing. Furthermore, the static link successfully reconstructed the three-dimensional scattering intensity distribution of the human body, and the dynamic link successfully obtained a micro-Doppler spectrum consistent with theoretical characteristics. These results jointly verify the effectiveness of the proposed method at all stages of the entire link, providing high-quality simulation data support for radar-based research on human behavior perception, action recognition, and posture reconstruction.

[0113] like Figure 6 Some embodiments of this application provide a text-command-driven human motion radar scattering simulation device, the device comprising: The semantic condition vector extraction module 1001 is used to fuse the semantic feature vector extracted from the text instruction with the preset action prior vector to obtain the semantic condition vector; The human skeleton motion extraction module 1002 is used to generate a human skeleton motion sequence based on a semantic condition vector; The human body 3D mesh extraction module 1003 is used to construct a human body 3D mesh sequence based on the human skeletal movement sequence; The radar echo data extraction module 1004 is used to calculate the radar echo data cube based on radar parameters and a three-dimensional human body mesh sequence. The multidimensional visualization module 1005 is used to complete multidimensional imaging visualization based on the radar echo data cube.

[0114] It should be noted that the text-command-driven human motion radar scattering simulation device provided in this embodiment is based on the same inventive concept as the text-command-driven human motion radar scattering simulation method described above. Therefore, the content of the text-command-driven human motion radar scattering simulation method described above is also applicable to the content of the text-command-driven human motion radar scattering simulation device in this embodiment, and will not be repeated here.

[0115] like Figure 7 This application also provides an electronic device, which includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described text instruction-driven human motion radar scattering simulation method of this disclosure.

[0116] Electronic devices can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0117] The electronic devices according to embodiments of this application will now be described in detail.

[0118] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute a text instruction-driven human motion radar scattering simulation method according to an embodiment of the present invention.

[0119] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0120] This invention also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions. The computer-executable instructions are used to cause a computer to execute the above-described text instruction-driven human motion radar scattering simulation method.

[0121] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0122] The embodiments described in this invention are intended to more clearly illustrate the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0123] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0124] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0126] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0127] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0131] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A text-command-driven method for simulating human motion radar scattering, characterized in that, The method includes: The semantic feature vector extracted from the text instruction is fused with the preset action prior vector to obtain the semantic condition vector; the text instruction includes action description information. Based on the semantic condition vector, a human skeleton motion sequence is generated; the human skeleton motion sequence includes the three-dimensional spatial position and rotation parameters of the human joints in each time frame; Based on the human skeletal movement sequence, a three-dimensional human mesh sequence is constructed; Based on the radar parameters and the three-dimensional mesh sequence of the human body, the radar echo data cube is calculated; Based on the radar echo data cube, multi-dimensional imaging visualization is completed; the generation of human skeletal action sequences based on the semantic condition vector includes: Generate a potential action sequence based on the semantic condition vector; Generate a sequence of motion parameters based on the potential action sequence and the semantic condition vector; Based on the sequence of motion parameters, a sequence of human skeletal movements is generated.

2. The text-command-driven human motion radar scattering simulation method according to claim 1, characterized in that, The step of generating a potential action sequence based on the semantic condition vector includes: Using the aforementioned semantic condition vector as semantic constraints and a standard Gaussian noise sequence as the diffusion starting point, DDIM sampling is performed step by step; After sampling is completed in all diffusion steps, the potential action sequence is obtained.

3. The text-command-driven human motion radar scattering simulation method according to claim 1, characterized in that, The step of generating a motion parameter sequence based on the potential action sequence and the semantic condition vector includes: The potential action sequence and the semantic condition vector are input into the temporal downsampling variational autocoder to obtain the normalized motion parameter sequence output by the temporal downsampling variational autocoder. The normalized motion parameter sequence is denormalized to obtain the motion parameter sequence; The step of generating a human skeletal movement sequence based on the motion parameter sequence includes: The global rotation, local rotation of body joints, and translation of root node for each frame are extracted from the motion parameter sequence. The global rotation and the local rotation of the body joints are respectively converted into global axis angles and body joint axis angles; By integrating the global axis angle, body joint axis angle, and root node translation of each frame, a human skeletal motion sequence is obtained.

4. The text-command-driven human motion radar scattering simulation method according to claim 1, characterized in that, The step of constructing a three-dimensional human body mesh sequence based on the human skeletal movement sequence includes: The human skeleton motion sequence and body shape parameters are input into the SMPL skinning function to obtain the human three-dimensional mesh sequence output by the SMPL skinning function.

5. The text-command-driven human motion radar scattering simulation method according to claim 1, characterized in that, The calculation of the radar echo data cube based on radar parameters and the three-dimensional human body mesh sequence includes: The human body 3D mesh sequence is traversed frame by frame to obtain the triangular element parameters corresponding to each frame mesh by performing triangular element subdivision on each frame mesh. Based on the parameters of the triangular surface element and the radar parameters, the complex scattering field of the triangular surface element is obtained; The complex scattering fields of all the triangular facets are summed to obtain the noiseless frequency domain echo of a single frame. Adding complex Gaussian white noise to the noiseless frequency domain echo of the single frame yields a noisy frequency domain echo of the single frame. A radar echo data cube is generated based on the noisy frequency domain echo of the single frame.

6. The text-command-driven human motion radar scattering simulation method according to claim 1, characterized in that, The step of completing multi-dimensional imaging visualization based on the radar echo data cube includes: The radar echo data cube is subjected to pulse compression processing and converted to the range domain to obtain a range image; Based on the range image, micro-Doppler frequency maps, range-Doppler maps, and range-time maps are generated respectively. Then, based on the range image and the MIMO array, a multi-dimensional imaging visualization is formed through a back projection algorithm.

7. A text-command-driven human motion radar scattering simulation device, characterized in that, The device includes: The semantic condition vector extraction module is used to fuse the semantic feature vector extracted from the text instruction with the preset action prior vector to obtain the semantic condition vector; the text instruction includes action description information. The human skeleton motion extraction module is used to generate a human skeleton motion sequence based on the semantic condition vector; the human skeleton motion sequence includes the three-dimensional spatial position and rotation parameters of the human joints in each time frame; The human body 3D mesh extraction module is used to construct a human body 3D mesh sequence based on the human skeletal movement sequence; The radar echo data extraction module is used to calculate the radar echo data cube based on radar parameters and the three-dimensional mesh sequence of the human body. The multi-dimensional visualization module is used to complete multi-dimensional imaging visualization based on the radar echo data cube; the step of generating a human skeletal action sequence based on the semantic condition vector includes: Generate a potential action sequence based on the semantic condition vector; Generate a sequence of motion parameters based on the potential action sequence and the semantic condition vector; Based on the sequence of motion parameters, a sequence of human skeletal movements is generated.

8. An electronic device, characterized in that, include: At least one control processor and a memory for communicatively connecting to the at least one control processor; The memory stores instructions that can be executed by the at least one control processor, which, when executed, enables the at least one control processor to perform a text instruction-driven human motion radar scattering simulation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to execute a text-instruction-driven human motion radar scattering simulation method as described in any one of claims 1 to 6.

Images