Virtual space generation device and virtual space generation method

The virtual space generation device addresses the need for customizable virtual spaces by using a language model and asset database to create tailored environments for training and maintenance, facilitating efficient skill acquisition through user-defined scenarios.

JP2025144748APending Publication Date: 2025-10-03HITACHI LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2024044583
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-03-21
Publication Date
2025-10-03

AI Technical Summary

Technical Problem

Existing technologies struggle to create multiple virtual spaces/metaverses necessary for comprehensive training and maintenance of facilities and equipment, particularly in railway maintenance, and lack a method for generating virtual spaces according to user requests.

Method used

A virtual space generation device and method utilizing a language model to acquire keywords for assets, an asset database to retrieve relevant assets, and a display control unit to output these assets in a virtual space based on user input, enabling the generation of customizable virtual environments for training and maintenance.

Benefits of technology

Enables efficient generation of virtual spaces tailored to user needs, allowing maintenance personnel to train in various scenarios and environments without manual arrangement of facilities and equipment, enhancing knowledge acquisition and skill development.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2025144748000001_ABST
    Figure 2025144748000001_ABST
Patent Text Reader

Abstract

To generate a virtual space according to a request from a user.SOLUTION: A virtual space generation device 100 comprises: an asset acquisition unit which acquires a keyword of an asset related to a content of work, by using a language model 121, based on language expression of the content of the work performed in a virtual space, refers to an asset database 130 in which the keyword and the asset are stored in association with each other, and acquires the asset which becomes an object of the work based on the acquired keyword; an asset arranging unit 113 which arranges the asset to the virtual space; and a display control unit 116 which outputs the asset to a display device.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to a virtual space generation device and a virtual space generation method for generating a virtual space (metaverse). [Background technology]

[0002] Advances in digital technology have made it possible to digitally recreate real-world objects such as buildings and facilities as high-resolution virtual objects in virtual spaces. Virtual spaces are also known as metaverse spaces. For example, in the case of railway maintenance, the facilities and equipment to be maintained, such as tracks, trains, electrical systems, signal systems, and stations, can be digitized and recreated in the metaverse. However, digitizing the maintenance work itself is difficult. One example of digitizing maintenance work itself is remote maintenance. Furthermore, the transfer of maintenance skills due to population decline has become an issue. If maintenance work training / simulation were possible in virtual spaces, it would be possible to pass on maintenance techniques and know-how.

[0003] When conducting work training in a virtual space, it is desirable to conduct training under a variety of circumstances and scenarios. For example, when it comes to maintenance and repair work on trains, tracks, signal systems, etc., training is required for a variety of circumstances and scenarios, and for a variety of work targets, depending on the needs of the maintenance personnel users. For this reason, it is necessary to build multiple virtual spaces / metaverses that combine different types of train and track structures, signal systems, platforms, work time periods, work seasons, work locations, etc. Trains and tracks include, for example, rails, sleepers, roadbeds, bridges, and tunnels. Work locations can also refer to, for example, urban areas, rural areas, and suburban areas.

[0004] One technology related to virtual spaces is described in Patent Document 1. Patent Document 1 discloses "a method for visualizing an interaction between an embodied artificial agent and digital content on an end-user display device of an electronic computing device, the method comprising the steps of creating an agent virtual environment having virtual environment coordinates, simulating the digital content in the agent virtual environment, simulating an embodied artificial agent in the agent virtual environment, enabling the embodied artificial agent to interact with the simulated digital content, and displaying the interaction between the embodied artificial agent and the digital content on the end-user display device." [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication 2021-531603 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology described in Patent Document 1 improves the interaction between objects in the metaverse and users. However, Patent Document 1 does not describe a method for building multiple metaverses. Building multiple virtual spaces / metaverses is not only necessary for training in railway maintenance work, but is also required for work and training in the development, design, production, and maintenance of other facilities, equipment, and devices. The present invention has been made in view of the above background, and an object of the present invention is to provide a virtual space generation device and a virtual space generation method that enable the generation of a virtual space according to the user's requests. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the virtual space generation device of the present invention includes an asset acquisition unit that uses a language model to acquire keywords for assets related to the content of work to be performed in a virtual space based on the linguistic expression of the content of the work, and refers to an asset database in which the keywords and assets are stored in association with each other to acquire the assets that are the subject of the work based on the acquired keywords; an asset placement unit that places the assets in the virtual space; and a display control unit that outputs the assets to a display device. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a virtual space generation device and a virtual space generation method that enable the generation of a virtual space according to the user's requests. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a functional block diagram of a virtual space generating device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a language model according to the present embodiment. [Figure 3] FIG. 2 is a diagram illustrating a language model according to the present embodiment. [Figure 4] FIG. 2 is a data configuration diagram of an asset database according to the present embodiment. [Figure 5] FIG. 10 is a diagram for explaining the virtual space generation process according to the present embodiment. [Figure 6] FIG. 2 is a diagram for explaining the arrangement of assets according to the present embodiment. [Figure 7] 10A to 10C are diagrams for explaining a three-dimensional model generation process according to the present embodiment. [Figure 8] FIG. 2 is a diagram for explaining a sound diffusion model according to the present embodiment. [Figure 9] FIG. 10 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the virtual space generation device according to the above-described embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] A virtual space generating device according to an embodiment of the present invention will be described below, taking railway maintenance work as an example. The virtual space generation device of this embodiment generates a virtual space for railway maintenance personnel to train in maintenance work. To explain in more detail, first, a maintenance personnel, who is a user of the virtual space generation device, inputs the details of the maintenance work to be trained and details about the facilities and equipment to be maintained into the virtual space generation device. The details about the facilities and equipment include, for example, names and functions. The virtual space generation device then uses a language model based on the input details to acquire keywords for the facilities and equipment (also referred to as assets) that will be the target of the maintenance work. Next, the virtual space generation device acquires 3D models of the facilities and equipment from an asset database based on the keywords and places them in the virtual space. Furthermore, upon receiving instructions from the maintenance personnel, the virtual space generation device rearranges the facilities and equipment (3D models).

[0011] By using such a virtual space generator, maintenance personnel can train in the virtual space of their choice. Maintenance personnel can generate a variety of virtual spaces by simply changing the work content entered into the virtual space generator, allowing them to train in maintenance work in a variety of situations with different work environments such as weather, seasons, and time of day, as well as different types, locations, and conditions of facilities and equipment.

[0012] <Configuration of the virtual space generation device> 1 is a functional block diagram of a virtual space generating device 100 according to this embodiment. The virtual space generating device 100 is a computer, and includes a control unit 110, a storage unit 120, and an input / output unit 180. User interface devices such as a display, keyboard, mouse, and microphone are connected to the input / output unit 180. The input / output unit 180 may include a communication device, enabling data transmission and reception with a head-mounted display.

[0013] <Virtual space generation device: memory unit> The storage unit 120 is configured to include storage devices such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive). The storage unit 120 stores a language model 121, an image diffusion model 122, an image generation model 123, a sound diffusion model 124, an asset database 130, a terminology library 140, asset placement data 150, and a program 128. Note that the various storage contents of the storage unit 120 may be stored in an external storage device such as a cloud server and read as needed.

[0014] <Memory section: Language model> The language model 121 in this embodiment is a language model related to railway maintenance work, and is a machine learning model that has already learned text related to railway maintenance work.

[0015] 2 is a diagram for explaining a language model 121 according to this embodiment. The language model 121 is trained and generated using learning data such as specifications (asset specifications) of facilities and equipment (assets) such as trains, tracks, and signal systems that are the targets of railway maintenance work, and a maintenance work training manual 211.

[0016] 3 is a diagram illustrating a language model 121 according to this embodiment. The language model 121 may be generated by transfer learning or fine-tuning a pre-trained language model 221 using training data such as an asset specification or a training manual 222. The image diffusion model 122, the image generation model 123, and the sound diffusion model 124 will be explained together with the model generation unit 115 described later.

[0017] <Memory section: Asset database> Returning to Fig. 1, we will continue to explain the storage unit 120. The asset database 130 is a database that stores information on assets, which are facilities and equipment related to railways. Examples of assets include trains, tracks, electrical systems, signal systems, platforms, and station buildings.

[0018] FIG. 4 is a data configuration diagram of the asset database 130 according to this embodiment. The asset database 130 is, for example, data in a table format, with one row (record) representing an asset. A record includes columns (attributes (attribute name and attribute value)) of identification information (denoted as "ID" in FIG. 4), name, category, type, use, function, and model. Here, the model is a three-dimensional model placed in a virtual space. The model is not only displayed in the virtual space, but also operates in response to operations by a maintenance worker. For example, the display of the model may change in response to button operations, or the model may return from a faulty state to a normal state by replacing a part.

[0019] The asset database 130 may include other attributes, such as location relationships between assets. Examples of location relationships include whether a train is on a track, whether a platform is located along a track, and the distance between a platform and a track. 4 is in a table format, but is not limited to this. The asset database 130 may be a non-relational database such as a key-value type, document type, or tree structure type.

[0020] <<Storage section: Terminology library, asset placement data, program>> Returning to FIG. 1, the description of the storage unit 120 will continue. The term library 140 includes meanings and related words in dictionaries and encyclopedias, meanings and related words in ontologies, and meanings and related words defined by users (maintenance personnel). The term library 140 is referenced by the asset acquisition unit 112, which will be described later. The asset placement data 150 stores position information of assets (facility / equipment) placed in the virtual space. The program 128 includes a description of the processing of the functional units included in the control unit 110, which will be described later.

[0021] <Virtual space generation device: control unit> The control unit 110 is configured to include a CPU (Central Processing Unit) and is equipped with a model driving unit 111, an asset acquisition unit 112, an asset placement unit 113, a change instruction reception unit 114, a model generation unit 115, and a display control unit 116. The control unit 110 may be configured to include a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc.

[0022] <Control unit: Virtual space generation processing> An overview of virtual space generation processing for arranging assets in an empty virtual space will be described before describing each of the functional units included in the control unit 110. Figure 5 is a diagram for explaining the virtual space generation processing according to this embodiment.

[0023] The asset acquisition unit 112 acquires text 231 including the work content of the maintenance work and the work target (asset) from the maintenance personnel. Next, the model driver 111 uses the language model 121 to acquire keywords 232 related to the text 231 .

[0024] Next, the asset acquisition unit 112 uses the term library 140 to add keywords further related to the keywords 232 to acquire keywords 233 . Next, the asset acquisition unit 112 refers to the asset database 130 and acquires the asset 234 related to the keyword 233 . Next, the asset placement unit 113 places the asset 234 in the virtual space 235 .

[0025] <Control unit: Model drive unit> The model driving unit 111 acquires asset keywords 232 related to the content of input text 231 based on the input text 231, using the language model 121. For example, the model driving unit 111 generates a prompt including the text "Please provide keywords for facilities and equipment related to the text shown below" and the input text 231, and inputs this to the language model 121 to acquire asset keywords 232 related to the content of the text 231.

[0026] <Control Unit: Asset Acquisition Unit> The asset acquisition unit 112 acquires an asset 234 related to the text input by the maintenance personnel. More specifically, the asset acquisition unit 112 outputs the text 231 input by the maintenance personnel to the model driving unit 111 and acquires asset keywords 232 related to the content of the text 231. Here, the asset acquisition unit 112 may refer to the term library 140 and add related words or words having the same meaning as the acquired keywords 232 to create new keywords 233.

[0027] Next, the asset acquisition unit 112 refers to the asset database 130 and acquires assets 234 related to the keyword 233. For example, the asset acquisition unit 112 acquires assets that include the keyword in the name, category, type, use, or function in the asset database 130 shown in Fig. 4. If the asset database 130 is a key-value type, it is sufficient to acquire assets that include the keyword in the value.

[0028] Keyword matching is not limited to an exact match, but may be a partial match, and assets containing keywords close to the keywords obtained using, for example, the k-nearest neighbor method may be acquired. The keyword may also be in the form of an attribute name and attribute value, or a key and value. In such cases, the asset acquisition unit 112 compares the value of the attribute name or key in the asset database 130 with the attribute value or value of the keyword. The key and value may also be considered as the attribute name and attribute value.

[0029] The asset acquisition unit 112 may acquire an asset when matching of multiple keywords is successful. For example, if the name is not included, the asset acquisition unit 112 may acquire an asset when matching of type and use or type and function is successful. Attribute names or keys may be weighted, and the asset acquisition unit 112 may acquire an asset when the sum of the weights of successfully matched attribute names or keys is equal to or greater than a predetermined value.

[0030] The text entered by the maintenance technician is not necessarily entered from a keyboard connected to the input / output unit 180. For example, the text may be text (language, linguistic expression) converted from voice input using a microphone connected to the input / output unit 180.

[0031] As described above, the virtual space generating device 100 includes an asset acquisition unit 112 that acquires asset keywords 232 related to the content of the work to be performed in the virtual space using a language model 121 based on the linguistic expression of the content of the work (see text 231). The asset acquisition unit 112 refers to an asset database 130 in which keywords and assets are stored in association with each other, and acquires an asset 234 that is the target of the work based on the acquired keyword 232.

[0032] The asset database 130 contains attribute values ​​associated with assets. The asset acquisition unit 112 acquires the asset 234 having the attribute value that has successfully matched with the acquired keyword.

[0033] In addition to the acquired keyword, which is the first keyword (see keyword 232), the asset acquisition unit 112 refers to the term library 140, which stores related words or meanings of the keyword, to acquire a second keyword (see keyword 233), which is a related word of the first keyword or a keyword with the same meaning. The asset acquisition unit 112 acquires the asset 234 of the attribute value that has successfully matched with the first keyword or the second keyword.

[0034] <Control unit: Asset placement unit, change instruction reception unit> The asset placement unit 113 places the assets acquired by the asset acquisition unit 112 in the virtual space. The asset placement unit 113 references the positional relationships between assets, for example, by referring to the asset database 130, and places the assets so as to satisfy those positional relationships. The asset placement unit 113 stores information on the positions and orientations of the placed assets in asset placement data 150.

[0035] The change instruction receiving unit 114 receives an instruction from a worker to change the placement of an asset. Generally, there can be multiple placements for the same asset. The change instruction from the worker may include the position and orientation of the asset after the change, or may be an instruction to change it randomly. Upon receiving the change instruction from the maintenance worker, the change instruction receiving unit 114 instructs the asset placement unit 113 to make the change. The asset placement unit 113 re-places the asset in accordance with the change instruction.

[0036] 6 is a diagram for explaining the placement of assets according to this embodiment. Trains, traffic lights, and platforms are placed as assets in virtual spaces 281, 282, and 283, but at different positions. The asset placement unit 113 places the same asset at different positions.

[0037] As described above, the virtual space generating device 100 includes the asset placement unit 113 that places the asset 234 in the virtual space 235 . The asset database 130 stores location relationships between assets 234 . The asset placement unit 113 places the asset 234 in accordance with the positional relationship.

[0038] The virtual space generating device 100 includes a change instruction receiving unit 114 that receives an instruction to change the layout of the assets 234 . The asset allocation unit 113 allocates the asset 234 in accordance with the allocation change instruction.

[0039] <Control unit: Model generation unit> Returning to Figure 1, the explanation of the control unit 110 continues. The model generation unit 115 generates a three-dimensional model of the asset (see the "model" attribute of the asset database 130 shown in Figure 4). If the asset database 130 does not contain an asset corresponding to the keyword 233 (see Figure 5), the asset acquisition unit 112 instructs the model generation unit 115 to generate a model corresponding to the keyword 233, and acquires the asset. The model generation unit 115 generates a three-dimensional model of the asset corresponding to the keyword 233, and stores it in the asset database 130.

[0040] FIG. 7 is a diagram for explaining the three-dimensional model generation process according to this embodiment. The model generation unit 115 uses the image diffusion model 122 based on the keywords 232 and 233 to generate an image 242 of an asset corresponding to the keywords 232 and 233. The image 242 is, for example, an image of the front or side of the asset. The image diffusion model 122 is a diffusion model that has learned images of railway-related assets, and is used to generate an image of an asset related to a keyword based on the keyword related to the asset.

[0041] Next, the model generation unit 115 uses the image generation model 123 based on the image 242 to generate an image 243 of the asset viewed from a different viewpoint. The image generation model 123 is a machine learning model that learns the angle of the viewpoint for an object and the image of the object viewed from that viewpoint. The image generation model 123 is consistent with the color and texture of the object in the input image and is used to generate an image of the object from a different viewpoint. The image generation model 123 may be an image generation model that has been fine-tuned to a railway asset using Low-Rank Adaptation (LoRA). Next, the model generation unit 115 executes a three-dimensional conversion process 244 using the image 243 as an input, and generates a three-dimensional model 245 .

[0042] The model generation unit 115 generates sounds related to the asset in addition to the image. More specifically, the model generation unit 115 generates sounds corresponding to the keywords 233 using the sound diffusion model 124. Examples of sounds include alarm sounds, sounds during tapping inspections, and sounds during asset operation.

[0043] 8 is a diagram illustrating the sound diffusion model 124 according to this embodiment. The sound diffusion model 124 is a diffusion model that has learned the correspondence between a keyword 251 and a spectrum 253 of a sound 252 corresponding to the keyword 251. The model generation unit 115 acquires a spectrum corresponding to the keywords 232 and 233 using the sound diffusion model 124, and generates the sound corresponding to the keywords 232 and 233 by converting the spectrum into sound.

[0044] <Control unit: Display control unit> Returning to FIG. 1, the description of the control unit 110 will be continued. The display control unit 116 refers to the asset placement data 150, and outputs and displays the asset 234 placed in the virtual space 235 on a display connected to the input / output unit 180. The display control unit 116 may also output and display the asset 234 on a head-mounted display.

[0045] As described above, the virtual space generating device 100 includes the model generating unit 115 that acquires the first image (see image 242), which is an image of the asset, from the keywords 232, 233 of the asset using the image diffusion model 122. The model generation unit 115 uses the image generation model 123 based on the first image to generate a second image (see image 243), which is an image of the asset viewed from a different direction from the first image. The model generation unit 115 performs a three-dimensional conversion process 244 based on the first image and the second image to generate a three-dimensional model 245 of the asset.

[0046] When there is no asset corresponding to the keyword 232, 233 in the asset database 130, the asset acquisition unit 112 instructs the model generation unit 115 to generate and acquire a three-dimensional model 245 related to the keyword. The model generation unit 115 generates the sound of the asset from the keyword of the asset using the sound diffusion model 124. The virtual space generating device 100 includes a display control unit 116 that outputs assets to a display device (for example, a display).

[0047] <Features of the virtual space generator> The virtual space generation device 100 provides a virtual space 235 for maintenance work training, in which facilities and equipment (assets) desired by the user, who is a maintenance worker, are placed. First, the maintenance worker inputs the training content and the facilities and equipment in a language (linguistic expression) such as text or voice. The virtual space generation device then acquires keywords 232 and 233 using the language model 121 and terminology library 140 based on the input language (see text 231). Next, the virtual space generation device 100 references the asset database 130 to acquire assets 234 and places them in the virtual space 235. The maintenance worker performs maintenance work training in the virtual space 235.

[0048] By using this virtual space generating device 100, maintenance personnel can generate a virtual space and train without having to go through the trouble of searching for and arranging the facilities and equipment they are training on. Furthermore, maintenance personnel can rearrange the facilities and equipment by issuing an instruction to change the asset layout to the virtual space generating device 100. Since maintenance personnel can train while changing the combination and layout of facilities and equipment, it is expected that they will be able to acquire knowledge and skills efficiently.

[0049] <<Variations>> Although several embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The virtual space generation device 100 in the above-described embodiments is intended for training in railway maintenance work, but is not limited to this. By constructing the language model 121, image diffusion model 122, image generation model 123, asset database 130, etc. according to the work, the virtual space generation device 100 can create virtual spaces for work and work training in the development, design, production, and maintenance of other facilities, equipment, and devices.

[0050] The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications are included in the scope and spirit of the invention described in this specification, etc., and are also included in the invention described in the claims and their equivalents.

[0051] <Hardware configuration> The virtual space generating device 100 according to the embodiment described above is realized by a computer 900 configured as shown in FIG. 9, for example. FIG. 9 is a hardware configuration diagram showing an example of a computer 900 that realizes the functions of the virtual space generating device 100 according to the embodiment described above. The computer 900 includes a CPU 901, a ROM 902, a RAM 903, an SSD 904, an input / output interface 905 (referred to as an input / output I / F (Interface) in FIG. 9), a communication interface 906 (referred to as a communication I / F in FIG. 9), and a media interface 907 (referred to as a media I / F in FIG. 9). The computer 900 may include an HDD (Hard Disc Drive) instead of the SSD 904, or may include an HDD in addition to the SSD 904.

[0052] The CPU 901 operates based on a program stored in the ROM 902 or the SSD 904, and performs control by the control unit 110 in Fig. 9. The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 starts up, programs related to the hardware of the computer 900, and the like.

[0053] The CPU 901 controls input devices 910 such as a mouse, keyboard, and microphone, and output devices 911 such as a display and printer, via an input / output interface 905. The CPU 901 acquires data from the input device 910 via the input / output interface 905, and outputs generated data to the output device 911.

[0054] The SSD 904 stores programs executed by the CPU 901 and data used by the programs. The communication interface 906 receives data from other devices (not shown) via a communication network and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.

[0055] The media interface 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads the program from the recording medium 912 onto the RAM 903 via the media interface 907 and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disk), a magneto-optical recording medium such as an MO (Magneto Optical disk), a magnetic recording medium, a conductive memory tape medium, a semiconductor memory, or the like.

[0056] For example, when the computer 900 functions as the virtual space generating device 100 according to the embodiment described above, the CPU 901 of the computer 900 executes a program 128 (see FIG. 1) loaded onto the RAM 903, thereby realizing the functions of the virtual space generating device 100. The CPU 901 reads the program from a recording medium 912 and executes it. Alternatively, the CPU 901 may read the program from another device via a communication network, or may install the program 128 from the recording medium 912 onto the SSD 904 and execute it. [Explanation of symbols]

[0057] 100 Virtual Space Generator 111 Model Drive Unit 112 Asset Acquisition Department 113 Asset Allocation Department 114 Change Instructions Reception Department 115 Model Generation Unit 116 Display control unit 121 language models 122 Image Diffusion Model 123 Image Generation Model 124 Sound diffusion model 128 programs 130 Asset Database 140 Terminology Library 150 Asset Placement Data

Claims

1. Based on the linguistic expression of the content of the work to be performed in the virtual space, a language model is used to acquire keywords for assets related to the content of the work; an asset acquisition unit that refers to an asset database in which the keywords and the assets are stored in association with each other and acquires the asset to be used in the work based on the acquired keyword; an asset placement unit that places the asset in the virtual space; a display control unit that outputs the asset to a display device. Virtual space generation device.

2. The asset database includes: including attribute values ​​relating to the asset; The asset acquisition unit Get assets whose attribute values ​​match successfully with the keywords you get The virtual space generating device according to claim 1 .

3. The asset acquisition unit In addition to the first keyword, which is the acquired keyword, a term library that stores related words or meanings of the keyword is referenced to acquire a second keyword, which is a related word of the first keyword or a keyword with the same meaning; Acquire assets whose attribute values ​​have been successfully matched with the first keyword or the second keyword. The virtual space generating device according to claim 2 .

4. Obtaining a first image, which is an image of the asset, from the keyword of the asset using an image diffusion model; generating a second image based on the first image using an image generation model, the second image being an image of the asset viewed from a different direction than the first image; a model generation unit that performs a three-dimensional conversion process based on the first image and the second image to generate a three-dimensional model of the asset; The asset acquisition unit When an asset corresponding to the keyword does not exist in the asset database, the model generation unit is instructed to generate and acquire a three-dimensional model related to the keyword. The virtual space generating device according to claim 1 .

5. The model generation unit Generate the sound of the asset using a sound diffusion model based on the keywords of the asset. The virtual space generating device according to claim 4 .

6. The asset database includes: Further storing a positional relationship between the assets; The asset allocation unit Arranging the assets according to the positional relationship The virtual space generating device according to claim 1 .

7. further comprising a change instruction receiving unit that receives an instruction to change the placement of the assets; The asset allocation unit The asset is arranged in accordance with the arrangement change instruction. The virtual space generating device according to claim 1 .

8. The virtual space generator acquiring keywords for assets related to the content of the work to be performed in the virtual space using a language model based on a linguistic expression of the content of the work; a step of referring to an asset database in which the keywords and the assets are stored in association with each other, and acquiring the assets to be used in the work based on the acquired keywords; placing the asset in the virtual space; outputting the asset to a display device. A method for generating virtual space.

Citation Information

Patent Citations

  • Machine Interaction

    JP2021531603A