Behavioral data generation device, behavioral data generation method, and program

Abstract

A behavioral data generation device according to one aspect of the present invention is provided with an acquisition unit and a generation unit. The acquisition unit acquires behavior description information describing a person's behavior, including action information relating to the person's actions, attribute information relating to the person's attributes, and spatial information relating to a space in which the person acts. The generation unit uses a machine learning model, including a latent diffusion model, to generate behavioral data that has a skeleton structure and that simulates the person's behavior on the basis of the behavior description information.

Description

Behavioral data generation device, behavioral data generation method, and program

[0001] The embodiments relate to a behavioral data generation device, a behavioral data generation method, and a program.

[0002] In order to infer the behavior of every person using a machine learning model, it is necessary to train the model on the continuous behavior of a variety of people in various environments. The datasets required for training are mainly created based on videos of actual people performing certain actions, but this requires costs in terms of manpower and time. Therefore, existing datasets are often used to train machine learning models.

[0003] There are existing datasets of behavioral data represented by a three-dimensional skeleton structure, but these datasets do not cover all human movements and do not reflect human attributes such as age, gender, and body type.

[0004] Conventionally, there are known techniques for generating behavioral data from behavioral descriptions using the above-mentioned data sets. However, these conventional techniques are unable to reflect the attributes of a person, or can only reflect some of the attributes. Therefore, it is desirable to generate behavioral data with a skeleton structure that reflects various conditions (e.g., the attributes of a person and the surrounding environment).

[0005] C. Guo, S. Zou, X. Zuo, S. Wang, W. Ji, X. Li, and L. Cheng, “Generating diverse and natural 3d human motions from text,” in Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5152-5161, June 2022. X. Chen, B. Jiang, W. Liu, Z. Huang, B. Fu, T. Chen, and G. Yu, “Executing your commands via motion diffusion in latent space,” in Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 18000-18010, 2023. G. Tevet, B. Gordon, A. Hertz, A. H. Bermano, and D. Cohen-Or, “Motionclip: Exposing human motion generation to clip space,” in Computer Vision-ECCV 2022: 17th Europe Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXII, pp. 358-374, Springer, 2022.

[0006] The present invention aims to generate behavior data with a skeleton structure that reflects a person's attributes and the surrounding environment.

[0007] A behavioral data generation device according to an embodiment of the present invention includes an acquisition unit and a generation unit. The acquisition unit acquires behavioral explanation information that describes a person's behavior, including behavior information related to the person's behavior, attribute information related to the person's attributes, and spatial information related to the space in which the person acts. The generation unit uses a machine learning model including a latent diffusion model to generate behavioral data with a skeleton structure that simulates the person's behavior based on the behavioral explanation information.

[0008] According to the present invention, it is possible to generate behavior data with a skeleton structure that reflects the attributes of a person and the surrounding environment.

[0009] FIG. 1 is a block diagram showing an example of an overview of inference processing by a behavior data generation device according to an embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the behavior data generation device of FIG. 1. FIG. 3 is a block diagram showing an example of the functional configuration of the behavior data generation device of FIG. 2. FIG. 4 is a block diagram showing an example of the functional configuration of a learning processing unit of FIG. 3. FIG. 5 is a configuration diagram showing an example of a machine learning model provided in a behavior data generation device according to an embodiment. FIG. 6 is a flowchart showing an example of first model learning processing in a behavior data generation device according to an embodiment. FIG. 7 is a flowchart showing an example of second model learning processing in a behavior data generation device according to an embodiment. FIG. 8 is a flowchart showing an example of inference processing in a behavior data generation device according to an embodiment.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] <Embodiment> [Configuration] FIG. 1 is a block diagram illustrating an example of an outline of inference processing by a behavioral data generation device according to an embodiment. When behavioral explanation information is input, the behavioral data generation device 100 of FIG. 1 can output behavioral data representing the content of the behavioral explanation information. The behavioral explanation information is, for example, a sentence (behavior explanation) that explains (describes) a person's behavior. Specific examples of behavioral explanation information include text information such as "In the park, a pregnant woman is walking unsteadily, looking for a place to sit" and "In a convenience store, a person with back pain is trying to reach a low-positioned item." Behavior data is, for example, data in which actions are assigned to a person's skeletal structure. If the content of the behavior explanation is classified into different types of information, the behavioral explanation information includes, for example, action information, attribute information, and spatial information.

[0012] The movement information is, for example, information about a person's movement. Specifically, the movement information is expressed as text information such as "a person walking around," "walking unsteadily," and "picking up a product."

[0013] Attribute information is, for example, information about the attributes of the person who is the subject of the action (person attributes). Person attributes include, for example, basic attributes and special attributes. Basic attributes include, for example, "age," "gender," "height," "body type," and "emotions." Special attributes correspond to the state of special (minority) attributes, for example, and include "has difficulty moving," "is pregnant," "has visual impairment," and "uses a cane." Specifically, attribute information is expressed as text information such as "pregnant woman" and "person who appears to have back pain." Note that attribute information includes at least one of basic attributes and special attributes.

[0014] The spatial information is, for example, information about the space in which a person acts. The spatial information is, for example, information about "stores," "factories," "hospitals," and "event venues." Specifically, the spatial information is expressed as text information such as "parks" and "convenience stores."

[0015] Fig. 2 is a block diagram showing an example of the hardware configuration of the behavioral data generation device of Fig. 1. The behavioral data generation device 100 of Fig. 2 includes a control circuit 101, a storage 102, a communication module 103, an interface 104, and a drive 105.

[0016] The control circuit 101 is a circuit that controls the overall components of the behavior data generation device 100. The control circuit 101 includes a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory). The ROM of the control circuit 101 stores programs used in various processes in the behavior data generation device 100. The CPU of the control circuit 101 controls the entire behavior data generation device 100 in accordance with the programs stored in the ROM of the control circuit 101. The RAM of the control circuit 101 is used as a working area for the CPU of the control circuit 101.

[0017] The storage 102 is configured by, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 102 stores information used in various processes in the behavior data generation device 100.

[0018] The communication module 103 is a circuit used to send and receive data to and from other devices.

[0019] The interface 104 is an interface mainly used for communication between a user and the control circuit 101. The interface 104 includes, for example, input devices and output devices. The input devices include a microphone, a camera, a touch panel, and operation buttons. The output devices include, for example, a speaker and a display.

[0020] The drive 105 is a device for reading software stored in the storage medium 106. The drive 105 includes, for example, a CD (Compact Disk) drive or a DVD (Digital Versatile Disk) drive.

[0021] The storage medium 106 is a medium that stores software electrically, magnetically, optically, mechanically, or chemically. The storage medium 106 may store programs for executing various processes in the behavioral data generation device 100.

[0022] The storage medium 106 may be a USB (Universal Serial Bus) memory. When the storage medium 106 is a USB memory, the storage medium 106 is connected to, for example, the communication module 103. In this case, the behavioral data generation device 100 does not need to include the drive 105.

[0023] Fig. 3 is a block diagram showing an example of the functional configuration of the behavioral data generation device of Fig. 2. The behavioral data generation device 100 of Fig. 3 includes an acquisition unit 110, a behavioral data generation unit 120, a model storage unit 130, an output unit 140, and a learning processing unit 150.

[0024] 3, the CPU of the control circuit 101 in FIG. 2 loads a program stored in the ROM of the control circuit 101 or the storage medium 106 into the RAM of the control circuit 101. The CPU of the control circuit 101 then interprets and executes the program loaded into the RAM of the control circuit 101. As a result, the behavioral data generation device 100 functions as a computer including an acquisition unit 110, a behavioral data generation unit 120, a model storage unit 130, an output unit 140, and a learning processing unit 150.

[0025] The acquisition unit 110 acquires the behavior explanation information through a user operation, for example, from an input device of the interface 104. The acquisition unit 110 outputs the behavior explanation information to the behavior data generation unit 120.

[0026] Furthermore, the acquisition unit 110 acquires learning information, for example, from the storage 102, in response to a user operation. The learning information includes, for example, video data for additional learning. The video data includes, for example, a person performing an action. Hereinafter, for ease of explanation, it is assumed that the video data includes a single person performing an action. The acquisition unit 110 outputs the learning information to the learning processing unit 150. Note that the learning information may include, for example, teaching data that associates behavior explanation information with behavior data.

[0027] The behavioral data generation unit 120 receives behavioral explanation information from the acquisition unit 110. The behavioral data generation unit 120 generates behavioral data with a skeleton structure that simulates a person's behavior based on the behavioral explanation information, using a machine learning model including a latent diffusion model. The machine learning model used by the behavioral data generation unit 120 is stored in, for example, the model storage unit 130. The behavioral data generation unit 120 outputs the generated behavioral data to the output unit 140.

[0028] Specifically, the behavioral data generation unit 120 generates behavioral data from behavioral explanation information using a machine learning model that uses a neural network that includes at least a variational autoencoder (VAE) and a latent diffusion model that applies a diffusion model technique to the latent space in the VAE.

[0029] More specifically, the behavior data generation unit 120 uses a text encoder to convert the behavior explanatory information into a format that can be embedded in a latent space (initial noise space) after the diffusion process (noise addition) in the latent diffusion model. Next, the behavior data generation unit 120 embeds the converted behavior explanatory information into the initial noise space. Next, the behavior data generation unit 120 uses a denoiser that performs the inverse diffusion process (denoising) in the latent diffusion model to denoise the initial noise space after embedding, and outputs the denoising result as a denoised space. Finally, the behavior data generation unit 120 uses a motion decoder in the VAE to output the behavior data from the denoised space.

[0030] The model storage unit 130 stores various machine learning models used by the behavioral data generation unit 120. The model storage unit 130 stores at least a VAE related to learning the features of behavioral data and a latent diffusion model related to learning the features of the latent space of the VAE.

[0031] In addition to the VAE and the latent diffusion model, the model storage unit 130 may store a text encoder used to embed data (behavior explanatory information) into the latent diffusion model, a multimodal large language model (MLLM) that estimates (extracts) behavior data from video data of learning information, and an MLLM that estimates behavior explanatory information from video data of learning information. One MLLM may estimate both behavior data and behavior explanatory information from video data of learning information.

[0032] The output unit 140 receives the behavioral data from the behavioral data generation unit 120. The output unit 140 outputs the behavioral data to an external device.

[0033] The learning processing unit 150 receives learning information from the acquisition unit 110. Based on the learning information, the learning processing unit 150 performs additional learning on the machine learning model stored in the model storage unit 130. The target of the additional learning is, for example, a VAE and a latent diffusion model. The specific configuration of the learning processing unit 150 will be described below with reference to FIG. 4.

[0034] Fig. 4 is a block diagram showing an example of the functional configuration of the learning processing unit 150 in Fig. 3. The learning processing unit 150 in Fig. 4 includes a behavioral data estimation unit 151, a first model learning unit 152, a behavior explanation information estimation unit 153, and a second model learning unit 154.

[0035] The behavioral data estimation unit 151 receives learning information from the acquisition unit 110. When the learning information is video data, the behavioral data estimation unit 151 estimates behavioral data of people appearing in the video data based on the video data. Specifically, the behavioral data estimation unit 151 generates (estimates) behavioral data based on the video data using MLLM. The behavioral data estimation unit 151 outputs the estimated learning behavioral data to the first model learning unit 152 and the second model learning unit 154.

[0036] The first model learning unit 152 receives behavioral data for learning from the behavioral data estimation unit 151. The first model learning unit 152 additionally learns the VAE using the behavioral data. The first model learning unit 152 causes the VAE to learn features of new behavioral data. Hereinafter, the first model will refer to the VAE.

[0037] Specifically, the first model training unit 152 uses the VAE's motion encoder to output probability distribution parameters from behavioral data (input behavioral data) and pseudo-samples the latent space using these parameters. Next, the first model training unit 152 uses the VAE's motion decoder to output behavioral data (output behavioral data) from the sampled latent space. Next, the first model training unit 152 calculates a reconstruction loss based on the input behavioral data and the output behavioral data, and a KL (Kullback-Leibler) divergence loss based on a probability distribution (approximate distribution) related to the motion encoder and a true distribution for the approximate distribution. Finally, the first model training unit 152 updates the VAE's parameters by reducing the calculated losses (reconstruction loss and KL divergence loss).

[0038] The behavior explanation information estimation unit 153 receives learning information from the acquisition unit 110. When the learning information is video data, the behavior explanation information estimation unit 153 estimates behavior explanation information of a person appearing in the video data based on the video data. Specifically, the behavior explanation information estimation unit 153 generates (estimates) behavior explanation information based on the video data using MLLM. The behavior explanation information estimation unit 153 outputs the estimated behavior explanation information for learning to the second model learning unit 154.

[0039] The second model training unit 154 receives behavioral data for training from the behavioral data estimation unit 151 and behavioral explanatory information for training from the behavioral explanatory information estimation unit 153. The behavioral data and behavioral explanatory information correspond to each other. The second model training unit 154 additionally trains a latent diffusion model using the behavioral data and behavioral explanatory information. The second model training unit 154 causes the latent diffusion model to learn the relationship between new behavioral data and behavioral explanatory information that are associated with each other. Hereinafter, the second model will refer to the latent diffusion model.

[0040] Specifically, the second model training unit 154 adds noise to a latent space (initial latent space) based on behavioral data (input behavioral data) and outputs the addition result as a noise space. Here, the initial latent space is generated by outputting probability distribution parameters from the input behavioral data using a VAE motion encoder and pseudo-sampling the latent space using these parameters. Next, the second model training unit 154 converts behavior explanatory information (movement information, attribute information, and spatial information) using a text encoder and embeds it into the noise space. Next, the second model training unit 154 denoises the embedded noise space using a denoiser of the latent diffusion model and outputs the denoised result as a denoised space. Next, the second model training unit 154 calculates the loss between the initial latent space and the denoised space. Finally, the second model training unit 154 updates the parameters of the latent diffusion model by reducing the calculated loss.

[0041] When the learning information is the above-described teaching data, the processing of the behavioral data estimation unit 151 and the processing of the behavior explanatory information estimation unit 153 in the learning processing unit 150 may be omitted. In this case, the first model learning unit 152 receives the behavioral data of the teaching data from the acquisition unit 110, and the second model learning unit 154 receives the teaching data.

[0042] The configuration of the behavioral data generation device has been described above. Next, an example of the configuration of the machine learning model included in the behavioral data generation device and the operation of the behavioral data generation device will be described.

[0043] FIG. 5 is a configuration diagram showing an example of a machine learning model included in the behavioral data generation device according to the embodiment. FIG. 5 shows a VAE as a first model, a latent diffusion model as a second model, and a text encoder. Below, an overview of the learning process of the first model (first model learning process), the learning process of the second model (second model learning process), and the inference process will be described with reference to FIG. 5. The first model learning process and the second model learning process are executed by the learning processing unit 150, and the inference process is executed by the behavioral data generation unit 120.

[0044] (Overview of First Model Learning Process) The first model learning process, for example, utilizes new behavioral data that is not included in a dataset of previously learned behavioral data. The first model learning process inputs input behavioral data, which is new behavioral data, to a motion encoder and outputs a latent space Z. Next, the first model learning process inputs the latent space Z to a motion decoder and outputs output behavioral data. Finally, the first model learning process updates the parameters of the motion encoder and motion decoder so that the input behavioral data and the output behavioral data match.

[0045] (Outline of Second Model Learning Process) After the first model learning process is performed, the second model learning process is performed. In the second model learning process, for example, the above-mentioned new behavioral data and behavior explanation information corresponding to this new behavioral data are used. The second model learning process performs a diffusion process, which is a process of adding noise to the above-mentioned latent space Z, T times to obtain a noise space Z Ｔ Next, the second model learning process generates the noise space Z Ｔ By embedding the action description information converted using a text encoder into the noise space Z Ｔ Next, the second model learning process generates a noise space Z ＴA denoising process (denoising), which is a process for removing noise from Z′, is performed T times to generate a denoised space Z′. Finally, the second model learning process updates the parameters of the denoiser that performs the denoising so that the latent space Z′, which is the initial latent space before noise is added (initial latent space), matches the denoised space Z′. Note that a U-Net is used for the denoising.

[0046] (Outline of inference process) The inference process uses action explanatory information corresponding to the action data to be generated. The inference process generates an embedded initial noise space by embedding action explanatory information converted using a text encoder into an initial noise space that mimics the noise space after the diffusion process using a latent diffusion model. Next, the inference process performs a denoising process on the embedded initial noise space to generate a denoised space. Finally, the inference process inputs the denoised space to a motion decoder and outputs the action data.

[0047] [Operation] Fig. 6 is a flowchart showing an example of a first model learning process in the behavioral data generation device according to the embodiment. The first model learning process is a process for updating the parameters of the VAE by additionally learning the VAE using new behavioral data. The process of the flowchart in Fig. 6 is started, for example, when a user executes the first model learning process.

[0048] (Step S110) The first model learning unit 152 receives input behavioral data. The input behavioral data is behavioral data for learning.

[0049] (Step S120) The first model learning unit 152 outputs parameters of a probability distribution from the input behavior data using a motion encoder.

[0050] (Step S130) The first model learning unit 152 pseudo-samples the latent space using the parameters.

[0051] (Step S140) The first model learning unit 152 uses the motion decoder to output output action data from the sampled latent space.

[0052] (Step S150) The first model learning unit 152 calculates the reconstruction loss and the KL divergence loss in the VAE.

[0053] (Step S160) The first model learning unit 152 updates the parameters of the VAE (first model) by reducing the reconstruction loss and the KL divergence loss. After step S160, the process of the flowchart in FIG. 6 ends.

[0054] 7 is a flowchart showing an example of a second model learning process in the behavioral data generation device according to the embodiment. The second model learning process is a process for updating parameters of the latent diffusion model by additionally learning the latent diffusion model using new behavioral data and teaching data that associates the new behavioral data. The process of the flowchart in FIG. 7 is started, for example, when a user executes the second model learning process.

[0055] (Step S210) The second model learning unit 154 receives input behavioral data. The input behavioral data is behavioral data for learning.

[0056] (Step S220) The second model learning unit 154 adds noise to the latent space (initial latent space) based on the input behavior data and outputs the addition result as a noise space.

[0057] (Step S230) The second model learning unit 154 receives the motion information, attribute information, and space information based on the input motion data. The motion information, attribute information, and space information correspond to the input motion data.

[0058] (Step S240) The second model learning unit 154 converts the motion information, attribute information, and spatial information using a text encoder and embeds them into a noise space.

[0059] (Step S250) The second model learning unit 154 performs denoising processing on the noise space after embedding using a denoiser, and outputs the denoising result as a denoised space.

[0060] (Step S260) The second model learning unit 154 calculates the loss between the initial latent space and the denoised space.

[0061] (Step S270) The second model learning unit 154 updates the parameters of the latent diffusion model (second model) by reducing the calculated loss. After step S270, the processing of the flowchart in FIG. 7 ends.

[0062] 8 is a flowchart showing an example of inference processing in the behavior data generation device according to the embodiment. The inference processing is processing for generating behavior data from behavior explanation information using a trained machine learning model. The processing of the flowchart in FIG. 8 starts, for example, when a user inputs behavior explanation information into the behavior data generation device 100.

[0063] (Step S310) The behavior data generation unit 120 receives behavior explanation information.

[0064] (Step S320) The behavior data generation unit 120 converts the behavior explanation information using a text encoder and embeds it into the initial noise space.

[0065] (Step S330) The behavioral data generation unit 120 performs denoising processing on the initial noise space after embedding using a denoiser, and outputs the denoising result as a denoised space.

[0066] (Step S340) The behavior data generation unit 120 outputs the behavior data from the denoised space using the motion decoder. After step S340, the processing of the flowchart in FIG. 8 ends.

[0067] The flowcharts described in the above embodiments are merely examples. The order of the steps in the flowcharts described in the embodiments may be changed to the extent possible, and other steps may be added.

[0068] [Effects] According to the above embodiment, the behavioral data generation device acquires behavioral explanation information that describes a person's behavior, including behavior information related to the person's behavior, attribute information related to the person's attributes, and spatial information related to the space in which the person acts, and generates behavioral data with a skeleton structure that simulates the person's behavior based on the behavioral explanation information using a machine learning model including a latent diffusion model.

[0069] Therefore, the behavioral data generation device according to the embodiment can generate behavioral data with a skeleton structure that reflects a person's attributes and the surrounding environment by taking into account attribute information and spatial information in addition to motion information. Specifically, the behavioral data generation device according to the embodiment can generate behavioral data that captures a person's characteristics better than conventional methods. Furthermore, the behavioral data generation device according to the embodiment can generate behavioral data that makes avatar behavior natural in environments constructed in VR (Virtual Reality) spaces (e.g., hospitals and world expo venues).

[0070] According to the behavior data generation device of the embodiment, behavior data with a skeleton structure that also reflects the attributes of a person and the surrounding environment can be generated from the following behavior description information. Examples of behavior description information are shown below.

[0071] ・A girl who cannot reach a high shelf in a supermarket ・An elderly person with a cane staggering along at an outdoor venue such as an expo ・A pregnant woman staggering along in a park, looking for a place to sit ・A blind person who is in danger of hitting a high obstacle while walking ・A person with back pain at a convenience store, trying to reach an item that is low ・A person in a wheelchair encountering stairs ・A tourist lying on the ground outside, unable to move ・A person with a broken bone walking with crutches at a hospital

[0072] Furthermore, the behavioral data generation device according to the embodiment may perform additional learning on the latent diffusion model, acquire teaching data that associates behavioral explanatory information with behavioral data, output a noise space in which noise is added to a latent space based on the behavioral data, embed the behavioral explanatory information converted using a text encoder into the noise space, denoise the noise space after embedding using a denoiser included in the latent diffusion model, and output a denoised space, calculate the loss between the latent space and the denoised space, and update the parameters of the denoiser by reducing the loss.

[0073] As a result, the behavioral data generation device according to the embodiment can reflect attribute information and spatial information through additional learning, even in a latent diffusion model that has been trained by associating only behavior information with behavior data.

[0074] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected components from the disclosed components. For example, if the problem can be solved and the effects can be obtained even if some components are removed from all the components shown in the embodiments, the configuration from which these components are removed can be extracted as an invention.

[0075] DESCRIPTION OF SYMBOLS 100... Behavioral data generation device 101... Control circuit 102... Storage 103... Communication module 104... Interface 105... Drive 106... Storage medium 110... Acquisition unit 120... Behavioral data generation unit 130... Model storage unit 140... Output unit 150... Learning processing unit 151... Behavioral data estimation unit 152... First model learning unit 153... Behavioral explanation information estimation unit 154... Second model learning unit

Claims

1. A behavioral data generation device comprising: an acquisition unit that acquires behavioral explanatory information that explains the behavior of a person, the behavioral explanatory information including behavioral information about the person's behavior, attribute information about the person's attributes, and spatial information about the space in which the person acts; and a generation unit that generates behavioral data with a skeleton structure that simulates the behavior of the person based on the behavioral explanatory information using a machine learning model including a latent diffusion model.

2. The behavioral data generation device according to claim 1, further comprising a learning processing unit that performs additional learning on the machine learning model, wherein the acquisition unit acquires teaching data that associates the behavior explanatory information with the behavior data, and the learning processing unit outputs a noise space by adding noise to a latent space based on the behavior data, embeds the behavior explanatory information converted using a text encoder into the noise space, denoises the noise space after embedding using a denoiser included in the latent diffusion model, and outputs a denoised space, calculates a loss between the latent space and the denoised space, and updates parameters of the denoiser by reducing the loss.

3. A behavioral data generation method comprising: acquiring behavioral explanatory information that explains the behavior of a person, the behavioral explanatory information including behavioral information about the person's behavior, attribute information about the person's attributes, and spatial information about the space in which the person acts; and generating behavioral data with a skeleton structure that simulates the behavior of the person based on the behavioral explanatory information using a machine learning model including a latent diffusion model.

4. A program that causes a computer to function as: a means for acquiring behavioral explanatory information that explains the behavior of a person, including behavioral information about the person's behavior, attribute information about the person's attributes, and spatial information about the space in which the person acts; and a means for generating skeletal structured behavioral data that simulates the behavior of the person based on the behavioral explanatory information, using a machine learning model including a latent diffusion model.

Images