System

Abstract

A system is provided.SOLUTION: Means for collecting, tagging, and classifying scenic images, means for training an image recognition model using the scenic images, means for generating a prototype of a virtual city using the trained image recognition model, and means for receiving a requirement of a user and reflecting the requirement in the prototype of the virtual city, A system comprising: means for generating a detailed 3D model; means for setting up an earthquake scenario and performing an earthquake simulation in a virtual city to provide escape routes and support points; means for generating a post-earthquake recovery support plan to calculate required resources and present a recovery plan; means for collecting and integrating real city information into a virtual city database; and means for updating real city planning and earthquake countermeasures based on the integrated data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern society requires effective urban planning and rapid reconstruction support in the event of natural disasters such as earthquakes. However, collecting and analyzing data in the real world requires a great deal of time and cost, making it difficult to respond quickly. Furthermore, the technology for smoothly integrating virtual and real-world urban planning remains immature. This presents challenges in optimizing evacuation routes in the event of a disaster and in formulating appropriate plans for reconstruction support. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: A means for collecting, tagging, and classifying landscape images, and a means for training an image recognition model using the collected landscape images. The system also includes a means for generating a virtual city prototype using the trained image recognition model. The system also includes a means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model. The system also includes a means for setting an earthquake scenario, running an earthquake simulation in a virtual city environment, and providing evacuation routes and support points. The system also provides a means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan. The system also includes a means for collecting real-world city information and integrating it into a virtual city database. Finally, the system also includes a means for updating real-world city plans and earthquake response measures based on the integrated data, thereby achieving effective urban planning and rapid reconstruction support for disasters such as earthquakes.

[0006] "Landscape images" refer to images of urban or natural landscapes.

[0007] "Tagging" refers to the act of assigning identifiable information labels to images or data.

[0008] "Classification" refers to the process of organizing collected data or images based on specific criteria or categories.

[0009] An "image recognition model" refers to a trained algorithm or neural network used to analyze image data and identify features.

[0010] "Prototype" refers to an initially designed prototype or conceptual model.

[0011] "User requirements" refers to the functions and specifications that users desire for a system or project.

[0012] "3D model" refers to computer graphics data that represents three-dimensional spatial data.

[0013] An "earthquake disaster scenario" refers to a scenario designed based on simulations that assume the circumstances under which an earthquake will occur and its impact.

[0014] "Earthquake simulation" refers to the process of simulating the effects of an earthquake in a virtual environment and analyzing the results.

[0015] An "evacuation route" refers to a route for safe evacuation in the event of a disaster.

[0016] "Support points" refer to locations and facilities where emergency support is provided in the event of a disaster.

[0017] A "reconstruction support plan" refers to a plan for effectively carrying out recovery work and support activities after a disaster.

[0018] "Resources" refer to the personnel, funds, materials, equipment, etc. required to achieve a specific purpose.

[0019] "City information" refers to geographical, social and economic data about cities.

[0020] A "database" refers to a collection of data that systematically organizes large amounts of information and makes it searchable.

[0021] "Urban planning" refers to the process and plan for formulating long-term plans for urban development and maintenance.

[0022] "Earthquake response measures" refer to specific response methods and plans for minimizing damage and quickly recovering from an earthquake disaster. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] The following system configuration and operation will be described as an embodiment of the invention: This system uses landscape images to build a virtual city and to respond to earthquake disasters and provide reconstruction support.

[0045] 1. Collecting landscape images and training the model

[0046] Collection of landscape images

[0047] User: Uploads images of urban and natural landscapes that he has taken to the system.

[0048] Terminal: The user's terminal sends the uploaded landscape image to the server.

[0049] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[0050] Image recognition model training

[0051] Server: Uses the stored landscape images to train an image recognition model such as a convolutional neural network (CNN).

[0052] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0053] 2. Building a Virtual City

[0054] Virtual city prototyping

[0055] Server: Uses trained image recognition models to generate terrain and infrastructure prototypes from landscape image data.

[0056] Server: This prototype will be designed to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[0057] Gathering and elaborating user requirements

[0058] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system.

[0059] Terminal: User input is sent to the server in real time.

[0060] Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements and exports it to the VR environment.

[0061] 3. Earthquake response simulation and reconstruction support

[0062] Setting up a disaster scenario

[0063] User: Set up a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[0064] Terminal: Sends scenario setting information to the server.

[0065] Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[0066] Creation of a recovery support plan

[0067] User: Submits a request for a post-disaster recovery plan.

[0068] Server: Analyzes the impact of the earthquake and calculates the necessary resources (e.g., building materials, personnel, and equipment). Generates an optimal reconstruction support plan and presents it to the user.

[0069] Device: The user checks the reconstruction assistance plan in a VR environment.

[0070] 4. Integrating virtual cities with the real world

[0071] Real-World Data Collection

[0072] Users: Enter real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[0073] Terminal: Sends collected information to the server.

[0074] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[0075] Providing integrated data and updating urban planning

[0076] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0077] Device: Provide users with up-to-date planning information and gather feedback.

[0078] User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[0079] For example, a user can upload landscape images they have taken, and the system will analyze them and generate a prototype of a virtual city. After that, the user can set up an earthquake scenario in the VR environment and check the generated simulation results, which will enable efficient earthquake response and reconstruction support.

[0080] The processing flow will be explained below.

[0081] Step 1:

[0082] User: Uploads landscape images to the system.

[0083] Terminal: Sends the uploaded landscape image to the server.

[0084] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[0085] Step 2:

[0086] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[0087] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0088] Step 3:

[0089] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[0090] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[0091] Step 4:

[0092] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[0093] Terminal: Sends the requirements entered by the user to the server in real time.

[0094] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[0095] Step 5:

[0096] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[0097] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[0098] Step 6:

[0099] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[0100] Terminal: Sends the configured earthquake scenario to the server.

[0101] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[0102] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[0103] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[0104] Step 7:

[0105] User: Requests the generation of a post-disaster recovery plan.

[0106] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[0107] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[0108] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[0109] Step 8:

[0110] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[0111] Terminal: Sends collected information to the server.

[0112] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[0113] Step 9:

[0114] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0115] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[0116] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[0117] The above are the processing steps of the specific program for carrying out the invention.

[0118] Example 1

[0119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Conventional urban planning and earthquake response systems have difficulty integrating and operating real-world data with virtual environments, resulting in a lack of integrated means for efficient urban planning, disaster simulation, and reconstruction support. Furthermore, it is difficult to update user-provided information in real time in the virtual city, making it difficult to effectively use these systems in situations where a rapid and accurate response is required.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0122] In this invention, the server includes means for collecting landscape images taken by users and tagging and classifying them, means for training an image recognition model such as a convolutional neural network using the landscape image data, means for generating a virtual city prototype using the trained image recognition model, means for inputting user requirements, means for converting the virtual city prototype into a detailed 3D model reflecting the user requirements, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for using the server to generate a post-earthquake reconstruction support plan, calculate the required resources, and present the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, and means for updating real-world city plans and earthquake response measures based on the integrated data. This enables information provided by users to be quickly and accurately reflected in the virtual city, enabling efficient urban planning, earthquake response, and reconstruction support.

[0123] "User" refers to a person who uploads landscape images to the system, inputs requirements, and checks and uses the simulation results and reconstruction plans.

[0124] "Terminal" refers to a device that a user connects to and operates the system, and is used to upload landscape images, input requirements, and check simulation results.

[0125] The "server" refers to the central processing unit that is the back-end computing resource that supports the entire system and performs tasks such as collecting images, tagging, classifying, learning, generating prototypes, running simulations, generating reconstruction assistance plans, and integrating data.

[0126] "Landscape images" refer to image data of urban and natural landscapes that users take and upload to the system.

[0127] "Tagging" refers to the process of categorizing collected landscape images by assigning keywords such as "mountain," "river," and "building."

[0128] "Classification" refers to the process of organizing tagged landscape images into categories and storing them in a database.

[0129] An "image recognition model" refers to a machine learning model trained to identify and classify landscape images using a convolutional neural network (CNN) or similar.

[0130] "Virtual city" refers to a virtual urban environment created based on landscape image data, generated using a trained image recognition model.

[0131] "Prototype" refers to an early model or structure of a virtual city, a temporary mockup before being refined with further user requirements and data.

[0132] "User requirements" refers to the various specific requests and specifications for the virtual city that the user inputs into the system.

[0133] "Detailed 3D model" refers to a detailed three-dimensional virtual city model that is generated as a result of reflecting user requirements.

[0134] "Earthquake scenario" refers to the conditions for an earthquake to occur (e.g., epicenter, seismic intensity, time period) set in a virtual city environment.

[0135] "Earthquake simulation" refers to the process of simulating the impact and evacuation routes of an earthquake occurring within a virtual city environment based on a set earthquake scenario.

[0136] An "evacuation route" refers to a route within a virtual city that is set up for safe evacuation in an earthquake simulation.

[0137] "Support points" refer to points or locations where evacuees can receive support in earthquake simulations.

[0138] "Reconstruction Support Plan" refers to a plan that includes the resources (e.g., building materials, personnel, and equipment) needed to rebuild a virtual city after a disaster.

[0139] "Real-world urban information" refers to data such as the current state of cities and ongoing construction projects in the real world.

[0140] A "virtual city database" refers to a database for centrally managing real city information and virtual city information.

[0141] "Urban planning" refers to plans and policies for the future development and maintenance of virtual and real cities.

[0142] "Earthquake response measures" refers to plans that include actions, procedures, and preparations to be taken in the event of an earthquake.

[0143] This invention is a system for constructing a virtual city using landscape images and for responding to earthquake disasters and supporting reconstruction efforts. How each step is carried out will be described in detail below.

[0144] Collecting landscape images and learning models

[0145] Collection of landscape images

[0146] 1. User: Takes images of urban or natural scenery with a smartphone or digital camera and uploads them to the system using a dedicated application.

[0147] 2. Device: The user's device (e.g., smartphone or PC) sends the uploaded landscape images to the server.

[0148] 3. Server: The received landscape images are stored in a database and tagged and classified. Tags are automatically assigned using an image recognition algorithm, such as "mountain," "river," and "building." Classification is based on these tags.

[0149] Image recognition model training

[0150] 1. Server: Using the stored landscape images, an image recognition model such as a convolutional neural network (CNN) is trained using a machine learning library such as TensorFlow or PyTorch.

[0151] 2. Server: As a result of the training, a reliable model for identifying landscape images is generated and stored in a database.

[0152] Building a Virtual City

[0153] Virtual city prototyping

[0154] 1. Server: Using trained image recognition models, prototypes of terrain and infrastructure are generated from landscape image data. For example, prototypes are created using game engines such as Unity or Unreal Engine.

[0155] 2. Server: This prototype is applied with algorithms to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[0156] Gathering and elaborating user requirements

[0157] 1. User: Uses a dedicated application or web interface to input specific requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting up residential areas) into the system.

[0158] 2. Terminal: Sends user input to the server in real time.

[0159] 3. Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment.

[0160] Earthquake response simulation and reconstruction support

[0161] Setting up a disaster scenario

[0162] 1. User: Set a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[0163] 2. Terminal: Sends the configured scenario information to the server.

[0164] 3. Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[0165] Creation of a recovery support plan

[0166] 1. User: Submits a request for a post-disaster recovery plan.

[0167] 2. Server: The system analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, equipment) using optimization models and simulation tools.

[0168] 3. Server: Generates the optimal reconstruction assistance plan and presents it to the user.

[0169] 4. Device: The user checks the reconstruction assistance plan in a VR environment.

[0170] Integrating virtual cities with the real world

[0171] Real-World Data Collection

[0172] 1. User: Enters real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[0173] 2. Terminal: Sends the collected information to the server.

[0174] 3. Server: Integrates real-world data into the virtual city database and centrally manages virtual city and real-world data.

[0175] Providing integrated data and updating urban planning

[0176] 1. Server: Updates real-world urban planning and disaster response measures based on the integrated data.

[0177] 2. Device: Provide users with up-to-date planning information and gather feedback.

[0178] 3. User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[0179] Specific examples

[0180] Users upload landscape images taken with their smartphones to the system through a dedicated application. The server classifies the images and generates a prototype of the virtual city using Unity. The user then inputs further detailed requirements, and the server generates a detailed 3D model based on those requirements. The user then sets up an earthquake scenario in the VR environment, and the server runs the earthquake simulation and provides evacuation routes and support points.

[0181] Prompt Sentence Examples

[0182] "Please explain in detail the process by which a user uploads landscape images they have taken, and the system classifies them to generate a prototype of a virtual city. Also, please explain with concrete examples the steps by which a user sets up an earthquake scenario in a VR environment and checks the results."

[0183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0184] Step 1:

[0185] Users take images of urban or natural scenery using a smartphone or digital camera and upload them to the system through a dedicated application. In this case, the input is the captured image data, and the output is the uploaded image data.

[0186] Step 2:

[0187] The terminal sends the scenery image uploaded by the user to the server. At this time, the input is the scenery image data present on the user's terminal, and the output is the image data sent to the server.

[0188] Step 3:

[0189] The server stores the received landscape images in a database and automatically tags and classifies them. Specifically, it uses an image recognition algorithm to assign tags such as "mountain," "river," and "building." The input is the landscape image data sent to the server, and the output is the tagged image data.

[0190] Step 4:

[0191] The server uses the stored landscape images to train an image recognition model (e.g., a convolutional neural network). Specifically, it uses machine learning libraries such as TensorFlow and PyTorch. In this case, the input is tagged landscape image data, and the output is the trained image recognition model.

[0192] Step 5:

[0193] The server uses a trained image recognition model to generate prototypes of terrain and infrastructure from landscape image data. Specifically, it uses game engines such as Unity or Unreal Engine. The input is the trained image recognition model and landscape image data, and the output is a prototype of a virtual city.

[0194] Step 6:

[0195] The server reflects earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.) in the generated prototype. At this time, the input is the virtual city prototype and the design standards, and the output is a prototype to which the earthquake-resistant standards have been applied.

[0196] Step 7:

[0197] Users input requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system using a dedicated application or a web interface. At this time, the input is the user's requirement data, and the output is the requirement data sent to the system.

[0198] Step 8:

[0199] The terminal transmits the user's input contents to the server in real time, where the input is the user's requirement data and the output is the requirement data transmitted to the server.

[0200] Step 9:

[0201] The server analyzes the user requirements, incorporates them into the prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment. The input is the user requirement data sent to the server, and the output is the detailed 3D model.

[0202] Step 10:

[0203] The user sets a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment. At this time, the input is the user's earthquake scenario setting data, and the output is the set scenario data.

[0204] Step 11:

[0205] The terminal transmits scenario setting information to the server. At this time, the input is the earthquake disaster scenario setting data of the user, and the output is the scenario data transmitted to the server.

[0206] Step 12:

[0207] The server runs the simulation in a virtual city environment and provides evacuation routes and support points. The input is the transmitted scenario data and a detailed 3D model, and the output is the simulation results.

[0208] Step 13:

[0209] A user sends a request for a post-disaster reconstruction assistance plan, where the input is the request data for the reconstruction assistance plan and the output is the request data sent to the server.

[0210] Step 14:

[0211] The server analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, and equipment) using optimization models and simulation tools. The inputs are simulation results and resource data, and the output is an optimal reconstruction assistance plan.

[0212] Step 15:

[0213] The server generates a reconstruction assistance plan and presents it to the user. At this time, the input is the optimal reconstruction assistance plan, and the output is the reconstruction assistance plan presented to the user.

[0214] Step 16:

[0215] The user checks the reconstruction support plan in the VR environment. The input is the reconstruction support plan presented to the user, and the output is the user's confirmation result.

[0216] Step 17:

[0217] Users input real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application, where the input is the real-world city information and the output is the city information entered into the application.

[0218] Step 18:

[0219] The terminal sends the collected information to the server, where the input is the city information entered into the application and the output is the city information sent to the server.

[0220] Step 19:

[0221] The server integrates real-world data into the virtual city database and manages the virtual city and real-world data in a unified manner. At this time, the input is the real-world city information sent to the server, and the output is the integrated database.

[0222] Step 20:

[0223] The server updates the actual city plans and disaster response measures based on the integrated data. At this time, the input is the integrated database, and the output is the updated city plans and disaster response measures.

[0224] Step 21:

[0225] The terminal provides users with up-to-date planning information and collects their feedback, where the input is updated urban planning and disaster response measures, and the output is user feedback.

[0226] Step 22:

[0227] Users can submit their opinions and make necessary corrections or suggestions based on the information provided. The input is the updated urban plan, disaster response measures, and the user's opinions, and the output is feedback data.

[0228] (Application example 1)

[0229] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] Modern factories and urban infrastructure are required to be more resilient to natural disasters such as earthquakes. In particular, factories with large-scale facilities and complex infrastructure layouts face the challenge of minimizing damage in the event of an earthquake while maintaining an efficient and safe layout. It is also important to develop a rapid and effective post-earthquake reconstruction support plan. To address these challenges, it is necessary to conduct simulations in virtual space using images of real-world factory interiors.

[0231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0232] In this invention, the server includes means for collecting, tagging, and classifying landscape images, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and incorporating them into the virtual city prototype to generate a detailed 3D model, means for setting earthquake scenarios and running earthquake simulations in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, means for updating real-world city plans and earthquake response measures based on the integrated data, means for collecting landscape images of real-world factories and simulating the arrangement of equipment and layout in a virtual space, and means for proposing optimal arrangements and layout changes to minimize the impact of earthquakes. This enables the proposal of safe and efficient layouts using landscape images of real factories.

[0233] "Landscape images" are images of various urban and natural landscapes.

[0234] "Tagging" refers to assigning labels such as "mountain," "river," and "building" to landscape images.

[0235] "Classification" refers to sorting tagged landscape images into their respective categories.

[0236] An "image recognition model" is an algorithm for analyzing landscape images and identifying their content.

[0237] A "virtual city" is a model of a city created on a computer based on landscape images and user requirements.

[0238] The "prototype" is the first prototype of the virtual city.

[0239] A "detailed 3D model" is a specific and precise three-dimensional virtual city model that reflects the user's requirements.

[0240] An "earthquake disaster scenario" sets out specific earthquake conditions, such as the epicenter, seismic intensity, and time period.

[0241] An "earthquake simulation" is a simulation based on an earthquake scenario set in a virtual city.

[0242] An "evacuation route" is a route for safe evacuation in the event of an earthquake.

[0243] A "support point" is a base set up to carry out support activities in the event of a disaster.

[0244] The "Reconstruction Support Plan" is a plan for urban reconstruction after the earthquake.

[0245] "Necessary resources" refers to the building materials, personnel, equipment, etc. required to implement the reconstruction assistance plan.

[0246] "City information" refers to data about real cities, such as ongoing construction projects and the state of existing infrastructure.

[0247] "Integration" means incorporating real city information into a virtual city database and managing it centrally.

[0248] "Scenery images inside the factory" refer to images taken of each area inside the factory.

[0249] "Facility and layout arrangement" refers to the arrangement of machines, aisles, work areas, etc. within the factory.

[0250] "Optimal location" means arranging the equipment and layout within the factory in the most effective way to minimize the impact of an earthquake.

[0251] The system configuration and operation are specifically described below as an embodiment of the invention. This system uses landscape images of the factory to propose safe and efficient layouts, and supports earthquake disaster response and reconstruction efforts.

[0252] System Configuration

[0253] The hardware and software required for this system are as follows:

[0254] Hardware

[0255] Factory robot with a mobile camera

[0256] High-resolution camera

[0257] Devices installed in actual factories (PCs, tablets, etc.)

[0258] VR headset (e.g. HTC Vive, Oculus Rift, etc.)

[0259] software

[0260] Image processing library (OpenCV)

[0261] Image recognition model (CNN using Keras)

[0262] 3D model generation library (Trimesh)

[0263] Physics simulation library (PyBullet)

[0264] Detailed explanation of operation

[0265] Collecting landscape images and learning models

[0266] 1. Collection of landscape images

[0267] The factory robot moves around the factory, taking pictures of each area with a high-resolution camera and sending them to a terminal. This image data is then stored on a server, where it is tagged and classified.

[0268] 2. Training the image recognition model

[0269] The server uses the collected landscape images to train a convolutional neural network (CNN) and generate a model for identifying landscape images. This trained model is then used to analyze the layout and equipment within the factory.

[0270] Virtual Factory Construction and Simulation

[0271] 3. Virtual Factory Prototype Generation

[0272] Using a trained image recognition model, a prototype of the factory's terrain and infrastructure is generated from landscape image data, and this prototype is then adapted to reflect design standards that take earthquake resistance into account.

[0273] 4. Gathering and elaborating user requirements

[0274] Users input their requirements for the factory layout and equipment placement into the system via a terminal. These requirements are analyzed by the server and incorporated into the prototype. Finally, a detailed 3D model is generated and exported to the VR environment.

[0275] 5. Earthquake response simulation and reconstruction support

[0276] Users can set up specific earthquake scenarios in the VR environment. The server then runs a simulation based on this scenario, providing evacuation routes and support points. It also analyzes post-earthquake reconstruction support plans, calculates the necessary resources, and presents an optimal reconstruction plan.

[0277] 6. Real-World Data Integration

[0278] Users input real-world information about the factory through terminals, and this data is integrated into a virtual factory database on the server. Based on this integrated data, the real-world factory layout and earthquake response measures are updated as needed.

[0279] Examples of specific examples and prompts

[0280] For example, a prototype is generated by having a factory robot take pictures of the factory interior and uploading the collected images to a server, after which users can use a VR headset to set up earthquake scenarios and check the simulation results.

[0281] An example of a prompt to be sent to a generative AI model is as follows:

[0282] plaintext

[0283] "Please recognize the locations of machines and aisles in the factory from this landscape image and generate a 3D model."

[0284] This enables the analysis of landscape images and the efficient generation and simulation of virtual factories.

[0285] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0286] Step 1:

[0287] The user operates a factory robot and takes pictures of the factory interior using a high-resolution camera.

[0288] Input: Real-world environment inside a factory

[0289] Output: Landscape image data

[0290] Specific operation: The user activates the factory robot and moves it around a designated area. During the movement, the camera continuously captures images of the scenery and stores them in the robot's internal storage.

[0291] Step 2:

[0292] The terminal receives landscape image data from the factory robot and uploads it to the server.

[0293] Input: Scenery image data sent from a factory robot

[0294] Output: Landscape images stored on the server

[0295] Specific operation: The terminal connects to the factory robot via the network and receives the captured image data. The received data is checked on the terminal and, if there are no problems, is uploaded to the server.

[0296] Step 3:

[0297] The server classifies and tags the uploaded landscape images.

[0298] Input: Uploaded landscape image data

[0299] Output: Tagged landscape image data

[0300] How it works: The server uses image recognition algorithms to analyze each image, detecting features in the image and automatically assigning tags such as "machine," "aisle," and "work area." The classified image data is then stored in a database on the server.

[0301] Step 4:

[0302] The server uses the landscape image data to train a convolutional neural network (CNN) model.

[0303] Input: tagged landscape image data

[0304] Output: A trained CNN model

[0305] Specific operation: The server trains the CNN model using landscape images in the database as learning data. Once the model is trained, it is saved after checking its image recognition accuracy.

[0306] Step 5:

[0307] Generate virtual factory prototypes using trained image recognition models.

[0308] Input: trained CNN model, landscape image data

[0309] Output: 3D prototype model of the virtual factory

[0310] How it works: The server inputs landscape image data into a CNN model to extract information about equipment and layout. Based on the extracted information, a virtual factory prototype is generated using 3D modeling software.

[0311] Step 6:

[0312] The user inputs requirements regarding the layout and equipment placement within the factory via a terminal.

[0313] Input: User requirements for layout and equipment placement

[0314] Output: Detailed layout requirements data

[0315] Specific operation: The user uses the interface on the terminal to input the desired equipment layout, aisle locations, etc. This requirement data is sent to the server in real time.

[0316] Step 7:

[0317] The server reflects the user's requirements and generates a detailed 3D model.

[0318] Input: User requirement data, 3D prototype model of virtual factory

[0319] Output: Detailed 3D model

[0320] How it works: The server analyzes the user's requirements data and incorporates them into a prototype model of the virtual factory. A 3D model with a detailed layout is generated and exported to the VR environment.

[0321] Step 8:

[0322] Users can set up specific earthquake scenarios in a VR environment and run simulations.

[0323] Input: Earthquake scenario (epicenter, seismic intensity, time period, etc.)

[0324] Output: Earthquake simulation results (evacuation routes, affected areas, etc.)

[0325] Specific operation: The user puts on a VR headset and uses the interface to set up an earthquake scenario. The server runs a simulation based on the set scenario, and the results are displayed as a VR scenario.

[0326] Step 9:

[0327] The server generates a post-earthquake reconstruction support plan and calculates the required resources.

[0328] Input: Earthquake simulation results

[0329] Output: Recovery aid plan, list of necessary resources

[0330] Specific operation: The server analyzes the simulation results and creates a reconstruction assistance plan. It calculates the necessary resources such as building materials, personnel, and equipment, and proposes an optimal reconstruction plan based on that.

[0331] Step 10:

[0332] Users collect real-world factory information via their terminals and send it to the server.

[0333] Input: Real-world factory information (ongoing construction projects, infrastructure status, etc.)

[0334] Output: Consolidated factory data

[0335] How it works: Users input data about ongoing construction projects and existing infrastructure status into their terminals and send it to the server, where it is integrated into the virtual factory database.

[0336] Step 11:

[0337] The server updates the actual factory layout and earthquake response measures based on the integrated data.

[0338] Input: Integrated factory data

[0339] Output: Updated factory layout and earthquake response plan

[0340] Specific operation: The server analyzes the integrated data and updates the latest factory layout and earthquake response measures. The updated contents are notified to the user via the terminal, and corrections and suggestions are made as necessary.

[0341] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0342] This invention relates to a system that uses landscape images to build a virtual city and provide disaster response and reconstruction support for earthquakes and other disasters, and also combines it with an emotion engine that recognizes the user's emotions. This system analyzes the user's emotions and optimizes the design of the virtual city and reconstruction plans based on those analyses.

[0343] 1. Collecting landscape images and training the model

[0344] Collection of landscape images

[0345] User: Uploads images of urban and natural landscapes that he has taken to the system.

[0346] Terminal: The user's terminal sends the uploaded landscape image to the server.

[0347] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[0348] Image recognition model training

[0349] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[0350] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0351] 2. Building a Virtual City

[0352] Virtual city prototyping

[0353] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[0354] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[0355] Gathering and elaborating user requirements

[0356] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[0357] Terminal: User input is sent to the server in real time.

[0358] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[0359] Generate detailed 3D models

[0360] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[0361] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[0362] 3. Earthquake response simulation and reconstruction support

[0363] Earthquake scenario setting and simulation

[0364] User: Set up a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[0365] Terminal: Sends the configured earthquake scenario to the server.

[0366] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[0367] Server: As a result of the simulation, it calculates the locations of evacuation routes and important support facilities and generates corresponding maps.

[0368] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[0369] Creation of a recovery support plan

[0370] User: Requests the generation of a post-disaster recovery plan.

[0371] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[0372] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[0373] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[0374] 4. Incorporating an Emotional Engine

[0375] emotion recognition

[0376] Users: Express emotions through facial expressions, voice, and text input while using the system.

[0377] Device: Captures the user's facial expression data with a camera, records voice data with a microphone, and collects text input data.

[0378] Server: Analyzes the data sent from the device and uses an emotion engine to recognize the user's emotions, such as relief, anxiety, satisfaction, etc.

[0379] Utilizing Emotional Data

[0380] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels a lot of anxiety, it provides a design and support plan that emphasizes safety.

[0381] On the device: Present the optimized design or plan to the user and collect feedback again.

[0382] 5. Integrating virtual cities with the real world

[0383] Real-World Data Collection

[0384] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[0385] Terminal: Sends collected information to the server.

[0386] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[0387] Providing integrated data and updating urban planning

[0388] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0389] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[0390] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[0391] For example, a user uploads landscape images they have taken, and the system identifies them to generate a prototype of a virtual city. The user then sets up an earthquake scenario in the VR environment, runs a simulation, and checks evacuation routes and support points. During this process, the system analyzes the user's emotional data and provides an optimized earthquake response plan based on their sense of security or anxiety.

[0392] The processing flow will be explained below.

[0393] Step 1:

[0394] User: Uploads landscape images to the system.

[0395] Terminal: Sends the uploaded landscape image to the server.

[0396] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[0397] Step 2:

[0398] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[0399] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0400] Step 3:

[0401] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[0402] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[0403] Step 4:

[0404] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[0405] Terminal: User input is sent to the server in real time.

[0406] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[0407] Step 5:

[0408] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[0409] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[0410] Step 6:

[0411] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[0412] Terminal: Sends the configured earthquake scenario to the server.

[0413] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[0414] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[0415] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[0416] Step 7:

[0417] User: Requests the generation of a post-disaster recovery plan.

[0418] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[0419] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[0420] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[0421] Step 8:

[0422] Users: Express emotions naturally while using the system. Express emotions while interacting with and operating the system through facial expressions, voice, text input, etc.

[0423] Device: Captures the user's facial expressions with a camera, records their voice with a microphone, and collects the text they type. This data is sent to the server in real time.

[0424] Step 9:

[0425] Server: Analyzes the data sent from the device and uses an emotion engine to recognize the user's emotions, such as relief, anxiety, satisfaction, etc.

[0426] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. If the user feels a lot of anxiety, the server will adjust the design and reconstruction plan to emphasize safety.

[0427] Step 10:

[0428] Server: Presents the optimized design or plan to the user and again collects feedback.

[0429] Device: Displays updated designs and plans to the user in the VR environment and sends feedback to the server.

[0430] User: Review the optimized designs and plans provided and provide any necessary feedback.

[0431] Step 11:

[0432] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[0433] Terminal: Sends collected information to the server.

[0434] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[0435] Step 12:

[0436] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0437] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[0438] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[0439] The above are the specific program processing steps in the system that combines the emotion engine. This invention makes it possible to design virtual cities and support reconstruction efforts that take into account the emotions of users.

[0440] Example 2

[0441] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0442] In modern society, disaster response and reconstruction support are important, especially in urban environments. However, conventional systems have had difficulty integrating real-world city data with virtual city data and providing optimal plans that take user emotions into account. There is also a need for an integrated platform that can simulate specific disaster scenarios in a virtual reality environment and generate optimal response measures while performing real-time emotion analysis.

[0443] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0444] In this invention, the server includes means for collecting, tagging, and classifying landscape images, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan, means for analyzing user emotions and optimizing the virtual city design and reconstruction plan based on the recognized emotion data, means for collecting real city information and integrating it into a virtual city database, and means for updating the real city plan and earthquake response measures based on the integrated data. This enables the system to integrate real city data and virtual city data in real time and generate optimal disaster response and reconstruction plans that take user emotions into consideration.

[0445] A "landscape image" is a digital image of a city, nature, or other landscape.

[0446] "Tagging" is the process of assigning labels to images that indicate their characteristics, such as "mountain," "river," or "building."

[0447] "Classification" is the process of separating tagged images into different categories.

[0448] An "image recognition model" is a machine learning model that analyzes patterns in landscape images and automatically identifies their features.

[0449] "Virtual City Prototype" is the creation of an early stage city model in virtual space based on collected landscape images.

[0450] "Detailed 3D model" refers to a detailed three-dimensional city model that includes specific design information based on user requirements.

[0451] An "earthquake disaster scenario" is a hypothetical scenario that simulates earthquakes and their associated impacts that may occur under specific conditions.

[0452] "Earthquake simulation" is a simulation that applies an earthquake scenario to a virtual urban environment to analyze evacuation routes and support points in the event of an earthquake.

[0453] A "support point" is the location of a base where important support activities are carried out in the event of a disaster.

[0454] A "reconstruction support plan" is a detailed plan that includes the resources needed to rebuild a city after a disaster and a plan to rebuild infrastructure.

[0455] An "emotion engine" is software that analyzes emotions from a user's facial expressions, voice, and text input.

[0456] A "virtual reality environment" is a digital environment that allows users to experience an immersive virtual space.

[0457] "City information" refers to data about real cities that shows the state of buildings under construction and existing infrastructure.

[0458] A "database" is a system for systematically storing and managing landscape images, virtual city models, emotional data, etc.

[0459] "Integration" refers to combining multiple different data sets in order to manage and use them in a centralized manner.

[0460] This invention relates to a system that uses landscape images to build a virtual city and provide disaster response and reconstruction support, as well as a system that combines an emotion engine that recognizes the user's emotions. The system analyzes the user's emotions and optimizes the design of the virtual city and reconstruction plans based on those analyses.

[0461] 1. Collecting landscape images and training the model

[0462] Collection of landscape images

[0463] Users: Users upload images of urban and natural landscapes taken with their smartphones or digital cameras, thereby collecting data on realistic urban environments.

[0464] Terminal: The user's terminal sends these landscape images to the server.

[0465] Server: The server stores the received landscape images in a database and tags and classifies them. Specific tags include "mountain," "river," and "building." This process uses image recognition services such as Google Cloud Vision API.

[0466] Image recognition model training

[0467] Server: The server uses landscape images stored in a database to train an image recognition model. Specific software used includes TensorFlow and Keras.

[0468] Server: As a result of the training, a model that can accurately identify landscape images is generated. This model is saved in a database and used in subsequent processing.

[0469] 2. Building a Virtual City

[0470] Virtual city prototyping

[0471] Server: Utilizing a trained image recognition model, the server generates a prototype of a virtual city by analyzing landscape image data stored in a database. It uses 3D modeling software such as Unity.

[0472] Server: Apply earthquake-resistant design standards, such as seismic strength and evacuation routes, to the prototype and make adjustments.

[0473] Gathering and elaborating user requirements

[0474] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, setting of residential areas) through a web form.

[0475] Terminal: Sends user-entered data to the server in real time.

[0476] Server: Analyzes the received requirements and incorporates them into the virtual city prototype.

[0477] Generate detailed 3D models

[0478] Server: Generate a detailed 3D model of the virtual city using 3D modeling tools such as Blender or Maya.

[0479] Server: The generated 3D models are stored in a database and exported to a VR environment (e.g., Oculus Rift compatible).

[0480] 3. Earthquake response simulation and reconstruction support

[0481] Earthquake scenario setting and simulation

[0482] User: Set up a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[0483] Device: Sends settings to the server via the VR headset.

[0484] Server: Using simulation tools such as Unity, we run earthquake simulations in a virtual city environment, and analyze the results to determine evacuation routes and the locations of important support points.

[0485] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[0486] Creation of a recovery support plan

[0487] User: Requests a post-disaster recovery plan from the system.

[0488] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.). This utilizes Unity's analysis functions.

[0489] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[0490] Device: After viewing the reconstruction support plan in the VR environment, the user sends specific feedback to the server.

[0491] 4. Incorporating an Emotional Engine

[0492] emotion recognition

[0493] User: Expresses emotions through facial expressions, voice, and text input while using the system.

[0494] Device: Captures facial expressions with a camera, records audio with a microphone, and collects text input data.

[0495] Server: Analyzes the collected data using an emotion recognition API (e.g., Microsoft Azure Emotion API) to determine the user's emotions. For example, it identifies "relief," "anxiety," "satisfaction," etc.

[0496] Utilizing Emotional Data

[0497] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels anxious, it provides a design with enhanced safety and a support plan.

[0498] On the device: Present the optimized design or plan to the user and collect feedback again.

[0499] 5. Integrating virtual cities with the real world

[0500] Real-World Data Collection

[0501] Users: Enter real-world city information through the application, including the status of buildings under construction and the state of existing infrastructure.

[0502] Terminal: Sends collected information to the server.

[0503] Server: Integrates real-world data into the virtual city database and centrally manages data on the virtual city and real city.

[0504] Providing integrated data and updating urban planning

[0505] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0506] Device: Provides updated information to users, notifying them of the latest urban planning information and earthquake response measures.

[0507] User: Checks the provided information, sends necessary feedback to the server, and considers specific measures and actions based on this information.

[0508] Examples of specific examples and prompts

[0509] For example, a user uploads landscape images they have taken, and the system identifies them to generate a prototype of a virtual city. The user then sets up an earthquake scenario in the VR environment and runs a simulation to check evacuation routes and support points. During this process, the system analyzes the user's emotional data and provides an optimized earthquake response plan based on their sense of security or anxiety.

[0510] Example prompt sentence:

[0511] Please upload this landscape image.

[0512] "Please enter the requirements for your virtual city."

[0513] "Set up an earthquake scenario."

[0514] "Use your camera and microphone to express your emotions."

[0515] "Please enter real city information."

[0516] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0517] Step 1: Collecting landscape images

[0518] Users: Users upload images of city and natural scenery taken with their smartphones or digital cameras.

[0519] Input: Landscape image file (e.g. JPEG, PNG format)

[0520] Terminal: The user's terminal sends the uploaded landscape image to the server.

[0521] Server: The server stores the received landscape images in a database. It also uses a Python script to automatically tag and classify the images as "mountains," "rivers," "buildings," etc. This process uses the Google Cloud Vision API.

[0522] Output: A tagged and classified landscape image database

[0523] How it works: Users provide cityscape images following the prompt "Please upload," and the server stores and automatically tags them.

[0524] Step 2: Training the image recognition model

[0525] Server: The server uses the landscape images stored in the database to train an image recognition model using TensorFlow and Keras.

[0526] Input: A tagged and classified landscape image database

[0527] Data processing: Extract features from image data and train a model using machine learning algorithms.

[0528] Output: A trained, highly accurate image recognition model

[0529] How it works: The server retrieves landscape images from the database and trains them with the TensorFlow library to generate an image recognition model.

[0530] Step 3: Prototype the virtual city

[0531] Server: The server uses a trained image recognition model to analyze landscape image data from the database and generate a prototype of the virtual city. It uses 3D modeling software such as Unity.

[0532] Input: trained image recognition model, landscape image database

[0533] Data processing: Based on landscape images, prototypes are generated using 3D modeling software and seismic design criteria are applied.

[0534] Output: A prototype of a seismically designed virtual city

[0535] How it works: The server recognizes buildings and natural elements from landscape images and builds an initial virtual city model using Unity or Blender.

[0536] Step 4: Gathering and elaborating user requirements

[0537] User: The user fills out a web form with the requirements for the virtual city, including the placement of public facilities, the design of the transport infrastructure, and the setting of residential areas.

[0538] Input: User-entered requirement data (web form)

[0539] Terminal: The user terminal sends inputs to the server in real time.

[0540] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[0541] Output: Prototype reflecting user requirements

[0542] Specific operation: The user enters data into a web form through a prompt that says, "Please enter the requirements for your virtual city." The server analyzes this data and reflects it in the prototype.

[0543] Step 5: Generate a detailed 3D model

[0544] Server: The server uses 3D modeling tools such as Blender or Maya to generate a detailed 3D model of the virtual city based on the user requirements.

[0545] Input: Prototype reflecting user requirements

[0546] Data manipulation: Use 3D tools to refine and perfect your prototype.

[0547] Output: A detailed 3D model of a virtual city

[0548] What it does: The server takes requirements from the user and generates a detailed 3D model in Blender or Maya.

[0549] Step 6: Setting up and simulating earthquake scenarios

[0550] User: Set a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[0551] Input: Earthquake scenario (VR environment)

[0552] Device: Sends settings to the server via the VR headset.

[0553] Server: Using simulation tools such as Unity, we run earthquake simulations in a virtual city environment and analyze the results.

[0554] Data processing: Analyze the simulation results and calculate the locations of evacuation routes and support points.

[0555] Output: Simulation results including evacuation routes and assistance points

[0556] Specific operation: The user puts on the VR headset and sets up the scenario according to the prompt, "Please set up the earthquake scenario." The server receives the data and runs the simulation.

[0557] Step 7: Generate a recovery plan

[0558] User: The user requests the system to generate a post-disaster recovery plan.

[0559] Input: Request (instructions to the system)

[0560] Server: Analyzes the impact of the earthquake based on the simulation results and calculates the required resources. Utilizing Unity's analysis functions.

[0561] Data processing: Calculate the necessary resources (building materials, personnel, equipment, etc.) and generate an efficient reconstruction support plan.

[0562] Output: Recovery assistance plan, infrastructure reconstruction plan, housing reconstruction plan

[0563] Specific operation: The user requests, "Please generate a post-disaster reconstruction support plan," and the server calculates the necessary resources and generates the support plan based on this.

[0564] Step 8: Emotion Recognition

[0565] User: The user expresses their emotions through facial expressions, voice, and text input while using the system.

[0566] Input: facial expressions, voice, text data

[0567] Device: Uses a camera and microphone to capture facial expressions and voice, and collects text input data.

[0568] Server: Analyze user emotions from collected data using an emotion recognition API (e.g., Microsoft Azure Emotion API).

[0569] Data processing: Facial expressions, voice, and text data are analyzed using an emotion engine to identify emotions (e.g., relief, anxiety).

[0570] Output: Recognized emotion data

[0571] Specific behavior: While using the system, the user follows the prompt: "Please use your camera and microphone to express your emotions." The server analyzes the captured data and identifies the emotions.

[0572] Step 9: Leverage sentiment data

[0573] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels anxious, it provides a design and support plan that enhances safety.

[0574] Input: Recognized emotion data, virtual city data

[0575] Data processing: Adjust and optimize designs and plans based on sentiment data.

[0576] Output: Optimized virtual city design and reconstruction support plan

[0577] Specific operation: The server takes in the emotional data, makes design changes, for example to increase the sense of security, and shows them to the user.

[0578] Step 10: Real-world data collection and integration

[0579] User: Uses the application to input real-world city information (status of buildings under construction, existing infrastructure, etc.).

[0580] Input: Real city information

[0581] Terminal: Sends collected information to the server.

[0582] Server: Integrates real-world data into the virtual city database and centrally manages real and virtual data.

[0583] Output: Integrated city database

[0584] Specific operation: The user inputs real city information according to the application's prompts, and the server receives it and integrates it with the virtual city data.

[0585] Step 11: Providing integrated data and updating urban plans

[0586] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0587] Input: Integrated City Database

[0588] Data processing: Analyze the integrated data and update real-world urban planning and disaster response measures.

[0589] Output: Updated urban planning information, earthquake response measures

[0590] Specific operation: The server analyzes the integrated data, generates the latest urban planning information and earthquake response measures, and notifies the user.

[0591] The specific actions at each step ensure that the entire system functions efficiently, enabling users to obtain optimal disaster response and recovery plans in real time.

[0592] (Application example 2)

[0593] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0594] Although there are systems that build virtual cities based on real-world landscape images and provide disaster response and reconstruction support, conventional systems are not optimized to take user emotions into account, making it difficult to provide designs and plans that provide the most peace of mind and satisfaction to users. Furthermore, virtual store designs are not optimized based on user emotions, making it difficult to provide the optimal shopping experience tailored to each individual user.

[0595] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting landscape images and tagging and classifying them, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model, means for setting an earthquake scenario and performing an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating required resources, and presenting the reconstruction plan, means for collecting real city information and integrating it into a virtual city database, means for updating real city plans and earthquake response measures based on the integrated data, means for analyzing user emotion data using an emotion recognition engine and optimizing the virtual city design and reconstruction plan based on the emotion data, and means for generating a virtual store based on landscape images uploaded by a user, adjusting the design and product placement of the virtual store based on the emotion data, and providing the user with the virtual store optimized in the virtual reality environment. This will enable disaster response and reconstruction support plans to be provided that take into account the user's emotions, and will also enable the creation of an optimal shopping experience in virtual stores.

[0596] "Landscape images" are photographs or video footage of natural or urban landscapes.

[0597] "Tagging" is the act of assigning keywords and classification information to data or images.

[0598] "Classification" is the act of separating data or information into specific categories or groups.

[0599] An "image recognition model" is a mathematical model that uses machine learning algorithms to analyze images and identify their content.

[0600] A "virtual city" is an urban environment generated by computer simulation.

[0601] A "prototype" is an early model or prototype of a new system or product.

[0602] "User requirements" refers to the functions and conditions that users desire for the system.

[0603] A "detailed 3D model" is a model of an object or environment that is recreated in detail in three-dimensional space.

[0604] An "earthquake disaster scenario" is a simulation scenario that assumes the occurrence of an earthquake or related disaster.

[0605] "Earthquake simulation" is a system that uses a computer to recreate earthquakes and related disasters and analyze their impact.

[0606] An "evacuation route" is a route that people can use to safely evacuate in the event of a disaster.

[0607] "Support points" are important locations where support activities are carried out during disasters.

[0608] A "reconstruction support plan" is a plan to support the reconstruction of a region after an earthquake or other disaster.

[0609] "Resources" are the resources and materials required to run a system or project.

[0610] "Real city information" refers to data and contextual information about real cities.

[0611] The "Virtual City Database" is a database system that manages and stores various data related to virtual cities.

[0612] An "emotion recognition engine" is a system that analyzes a user's facial expressions, voice, text input, etc. to identify their emotions.

[0613] A "virtual store" is a virtual store built on the Internet or in a virtual reality space.

[0614] "Design" is the act of designing objects or environments with shapes and structures that have specific purposes and functions.

[0615] "Product placement" refers to the act of deciding how to place products in a store.

[0616] A "virtual reality environment" is a three-dimensional space generated using computer technology in which a user can act or experience something.

[0617] This system uses landscape images to build a virtual city, recognizes the user's emotions, and provides optimized designs and plans based on those emotions. This system is targeted at disaster response and reconstruction support, as well as virtual store generation and optimization.

[0618] System configuration

[0619] This system uses the following hardware and software:

[0620] Hardware

[0621] Smartphone (camera and microphone for capturing landscape images and emotion recognition)

[0622] Smart glasses (optional, for displaying virtual reality environments and emotion recognition)

[0623] Server (data processing and storage)

[0624] software

[0625] Image recognition model (convolutional neural network)

[0626] Emotion recognition engine (machine learning model)

[0627] Virtual Reality (VR) Engine

[0628] System Operation

[0629] 1. Collecting and tagging landscape images

[0630] Users use their smartphones to take pictures of urban or natural landscapes and upload them to the system. The server receives the uploaded images and tags and classifies them, adding tags such as "mountain," "river," and "building."

[0631] 2. Training the image recognition model

[0632] The server trains an image recognition model using landscape images stored in a database, using a convolutional neural network (CNN) to generate a highly accurate model for identifying landscape images.

[0633] 3. Prototype generation of virtual city

[0634] Using a trained image recognition model, the system analyzes landscape image data to generate a prototype of a virtual city, taking into account earthquake resistance standards and evacuation routes.

[0635] 4. Reflecting user requirements and generating detailed 3D models

[0636] Users input the requirements for a virtual city into the system, such as the layout of public facilities, the design of transportation infrastructure, the setting of residential areas, etc. The server analyzes the received requirements, incorporates them into the virtual city prototype, and generates a detailed 3D model.

[0637] 5. Earthquake Disaster Simulation and Generation of Recovery Support Plans

[0638] Based on the earthquake scenario set by the user, the server runs an earthquake simulation in a virtual city environment. Based on the simulation results, it calculates evacuation routes and support points, and generates a reconstruction support plan. It also calculates the necessary resources (building materials, personnel, equipment, etc.).

[0639] 6. Emotion Recognition and Optimization

[0640] When using the system, the camera and microphone on the smartphone or smart glasses are used to collect the user's facial expressions and speech. An emotion recognition engine analyzes this data and identifies the user's feelings of relief, anxiety, satisfaction, etc. The server then uses the recognized emotion data to optimize the design of the virtual city and reconstruction plans.

[0641] 7. Virtual store generation and optimization

[0642] The server generates a virtual store based on landscape images uploaded by users. The design and product layout of the virtual store are adjusted based on emotional data from users while using the system. The optimized virtual store is provided to users in a virtual reality environment, where they can experience and shop.

[0643] Prompt Sentence Examples

[0644] "Log in to this virtual store app and upload a landscape image taken with your smartphone. A virtual store will be generated based on that image. Next, walk around the store freely and look at the products. Actively express emotions such as joy, excitement, or anxiety when you pick up a particular product. The emotion recognition engine analyzes these emotions and suggests optimal store design and product placement."

[0645] The above is an embodiment of the present invention. This system makes it possible to provide disaster response and reconstruction support plans that take into account the user's emotions, and also to realize an optimal shopping experience in a virtual store.

[0646] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0647] Step 1:

[0648] Collecting and tagging landscape images

[0649] Users use their smartphones to take images of urban or natural landscapes and upload them to the system. The server receives the uploaded images and tags and classifies them. The input is the landscape image taken by the user, and the output is the tagged and classified landscape image. Specifically, the server uses a convolutional neural network (CNN) to extract image features and assign tags such as "mountain," "river," and "building."

[0650] Step 2:

[0651] Image recognition model training

[0652] The server trains an image recognition model using landscape images stored in a database. The input is tagged and classified landscape images, and the output is a trained image recognition model. Specifically, the server trains a CNN using a large landscape image dataset, which generates a highly accurate model for accurately identifying landscape images.

[0653] Step 3:

[0654] Virtual city prototyping

[0655] The server uses a trained image recognition model to analyze input landscape image data and generate a prototype of the virtual city. The output is an initial prototype of the virtual city. Specifically, the server extracts features from the landscape image data, generates 3D models of buildings and terrain, and uses these to create the basic structure of the virtual city.

[0656] Step 4:

[0657] Reflecting user requirements and generating detailed 3D models

[0658] Users input their requirements for a virtual city into the system, including the layout of public facilities, the design of transportation infrastructure, and the establishment of residential areas. The server receives this information, analyzes it, and applies it to a prototype of the virtual city, generating a detailed 3D model. The input is the user's requirements, and the output is a 3D model based on the user's requirements. Specifically, the server adjusts the layout of the prototype based on the conditions entered by the user and runs an algorithm to generate a detailed model.

[0659] Step 5:

[0660] Earthquake disaster simulation and generation of reconstruction support plans

[0661] The server runs an earthquake simulation in a virtual city environment, using an earthquake scenario set by the user as input. The input is the earthquake scenario, and the output is evacuation routes and support points based on the simulation results. The server analyzes the simulation results, calculates the resources required to generate a post-earthquake reconstruction support plan, and presents the reconstruction plan. Specifically, it simulates the impact within the virtual city based on parameters such as the location, time, and seismic intensity of the earthquake, and identifies evacuation routes and key points.

[0662] Step 6:

[0663] Emotion Recognition and Optimization

[0664] When a user uses the system, the system collects their facial expressions and speech using a camera and microphone installed on their smartphone or smart glasses. The input is the user's facial expression and voice data, and the output is analyzed emotional data. The server analyzes this data using an emotion recognition engine to identify the user's feelings of relief, anxiety, satisfaction, etc. Furthermore, it optimizes the design and reconstruction plan of the virtual city based on the emotional data. Specifically, it runs an algorithm to apply a design and layout that makes the user feel at ease.

[0665] Step 7:

[0666] Virtual store generation and optimization

[0667] The server generates a virtual store based on landscape images uploaded by users. The input is landscape images taken by users and emotional data, and the output is an optimized virtual store design. The server adjusts the virtual store design and product layout based on the emotional data. The optimized virtual store is provided to users in a virtual reality environment, where they can experience and shop. Specifically, the system analyzes emotional data in real time as users browse products, and sequentially optimizes product layout and design.

[0668] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0669] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0670] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0671] [Second embodiment]

[0672] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0673] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0674] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0675] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0676] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0677] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0678] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0679] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0680] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0681] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0682] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0683] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0684] The following system configuration and operation will be described as an embodiment of the invention: This system uses landscape images to build a virtual city and to respond to earthquake disasters and provide reconstruction support.

[0685] 1. Collecting landscape images and training the model

[0686] Collection of landscape images

[0687] User: Uploads images of urban and natural landscapes that he has taken to the system.

[0688] Terminal: The user's terminal sends the uploaded landscape image to the server.

[0689] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[0690] Image recognition model training

[0691] Server: Uses the stored landscape images to train an image recognition model such as a convolutional neural network (CNN).

[0692] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0693] 2. Building a Virtual City

[0694] Virtual city prototyping

[0695] Server: Uses trained image recognition models to generate terrain and infrastructure prototypes from landscape image data.

[0696] Server: This prototype will be designed to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[0697] Gathering and elaborating user requirements

[0698] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system.

[0699] Terminal: User input is sent to the server in real time.

[0700] Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements and exports it to the VR environment.

[0701] 3. Earthquake response simulation and reconstruction support

[0702] Setting up a disaster scenario

[0703] User: Set up a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[0704] Terminal: Sends scenario setting information to the server.

[0705] Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[0706] Creation of a recovery support plan

[0707] User: Submits a request for a post-disaster recovery plan.

[0708] Server: Analyzes the impact of the earthquake and calculates the necessary resources (e.g., building materials, personnel, and equipment). Generates an optimal reconstruction support plan and presents it to the user.

[0709] Device: The user checks the reconstruction assistance plan in a VR environment.

[0710] 4. Integrating virtual cities with the real world

[0711] Real-World Data Collection

[0712] Users: Enter real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[0713] Terminal: Sends collected information to the server.

[0714] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[0715] Providing integrated data and updating urban planning

[0716] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0717] Device: Provide users with up-to-date planning information and gather feedback.

[0718] User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[0719] For example, a user can upload landscape images they have taken, and the system will analyze them and generate a prototype of a virtual city. After that, the user can set up an earthquake scenario in the VR environment and check the generated simulation results, which will enable efficient earthquake response and reconstruction support.

[0720] The processing flow will be explained below.

[0721] Step 1:

[0722] User: Uploads landscape images to the system.

[0723] Terminal: Sends the uploaded landscape image to the server.

[0724] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[0725] Step 2:

[0726] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[0727] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0728] Step 3:

[0729] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[0730] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[0731] Step 4:

[0732] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[0733] Terminal: Sends the requirements entered by the user to the server in real time.

[0734] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[0735] Step 5:

[0736] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[0737] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[0738] Step 6:

[0739] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[0740] Terminal: Sends the configured earthquake scenario to the server.

[0741] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[0742] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[0743] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[0744] Step 7:

[0745] User: Requests the generation of a post-disaster recovery plan.

[0746] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[0747] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[0748] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[0749] Step 8:

[0750] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[0751] Terminal: Sends collected information to the server.

[0752] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[0753] Step 9:

[0754] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[0755] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[0756] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[0757] The above are the processing steps of the specific program for carrying out the invention.

[0758] Example 1

[0759] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0760] Conventional urban planning and earthquake response systems have difficulty integrating and operating real-world data with virtual environments, resulting in a lack of integrated means for efficient urban planning, disaster simulation, and reconstruction support. Furthermore, it is difficult to update user-provided information in real time in the virtual city, making it difficult to effectively use these systems in situations where a rapid and accurate response is required.

[0761] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0762] In this invention, the server includes means for collecting landscape images taken by users and tagging and classifying them, means for training an image recognition model such as a convolutional neural network using the landscape image data, means for generating a virtual city prototype using the trained image recognition model, means for inputting user requirements, means for converting the virtual city prototype into a detailed 3D model reflecting the user requirements, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for using the server to generate a post-earthquake reconstruction support plan, calculate the required resources, and present the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, and means for updating real-world city plans and earthquake response measures based on the integrated data. This enables information provided by users to be quickly and accurately reflected in the virtual city, enabling efficient urban planning, earthquake response, and reconstruction support.

[0763] "User" refers to a person who uploads landscape images to the system, inputs requirements, and checks and uses the simulation results and reconstruction plans.

[0764] "Terminal" refers to a device that a user connects to and operates the system, and is used to upload landscape images, input requirements, and check simulation results.

[0765] The "server" refers to the central processing unit that is the back-end computing resource that supports the entire system and performs tasks such as collecting images, tagging, classifying, learning, generating prototypes, running simulations, generating reconstruction assistance plans, and integrating data.

[0766] "Landscape images" refer to image data of urban and natural landscapes that users take and upload to the system.

[0767] "Tagging" refers to the process of categorizing collected landscape images by assigning keywords such as "mountain," "river," and "building."

[0768] "Classification" refers to the process of organizing tagged landscape images into categories and storing them in a database.

[0769] An "image recognition model" refers to a machine learning model trained to identify and classify landscape images using a convolutional neural network (CNN) or similar.

[0770] "Virtual city" refers to a virtual urban environment created based on landscape image data, generated using a trained image recognition model.

[0771] "Prototype" refers to an early model or structure of a virtual city, a temporary mockup before being refined with further user requirements and data.

[0772] "User requirements" refers to the various specific requests and specifications for the virtual city that the user inputs into the system.

[0773] "Detailed 3D model" refers to a detailed three-dimensional virtual city model that is generated as a result of reflecting user requirements.

[0774] "Earthquake scenario" refers to the conditions for an earthquake to occur (e.g., epicenter, seismic intensity, time period) set in a virtual city environment.

[0775] "Earthquake simulation" refers to the process of simulating the impact and evacuation routes of an earthquake occurring within a virtual city environment based on a set earthquake scenario.

[0776] An "evacuation route" refers to a route within a virtual city that is set up for safe evacuation in an earthquake simulation.

[0777] "Support points" refer to points or locations where evacuees can receive support in earthquake simulations.

[0778] "Reconstruction Support Plan" refers to a plan that includes the resources (e.g., building materials, personnel, and equipment) needed to rebuild a virtual city after a disaster.

[0779] "Real-world urban information" refers to data such as the current state of cities and ongoing construction projects in the real world.

[0780] A "virtual city database" refers to a database for centrally managing real city information and virtual city information.

[0781] "Urban planning" refers to plans and policies for the future development and maintenance of virtual and real cities.

[0782] "Earthquake response measures" refers to plans that include actions, procedures, and preparations to be taken in the event of an earthquake.

[0783] This invention is a system for constructing a virtual city using landscape images and for responding to earthquake disasters and supporting reconstruction efforts. How each step is carried out will be described in detail below.

[0784] Collecting landscape images and learning models

[0785] Collection of landscape images

[0786] 1. User: Takes images of urban or natural scenery with a smartphone or digital camera and uploads them to the system using a dedicated application.

[0787] 2. Device: The user's device (e.g., smartphone or PC) sends the uploaded landscape images to the server.

[0788] 3. Server: The received landscape images are stored in a database and tagged and classified. Tags are automatically assigned using an image recognition algorithm, such as "mountain," "river," and "building." Classification is based on these tags.

[0789] Image recognition model training

[0790] 1. Server: Using the stored landscape images, an image recognition model such as a convolutional neural network (CNN) is trained using a machine learning library such as TensorFlow or PyTorch.

[0791] 2. Server: As a result of the training, a reliable model for identifying landscape images is generated and stored in a database.

[0792] Building a Virtual City

[0793] Virtual city prototyping

[0794] 1. Server: Using trained image recognition models, prototypes of terrain and infrastructure are generated from landscape image data. For example, prototypes are created using game engines such as Unity or Unreal Engine.

[0795] 2. Server: This prototype is applied with algorithms to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[0796] Gathering and elaborating user requirements

[0797] 1. User: Uses a dedicated application or web interface to input specific requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting up residential areas) into the system.

[0798] 2. Terminal: Sends user input to the server in real time.

[0799] 3. Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment.

[0800] Earthquake response simulation and reconstruction support

[0801] Setting up a disaster scenario

[0802] 1. User: Set a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[0803] 2. Terminal: Sends the configured scenario information to the server.

[0804] 3. Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[0805] Creation of a recovery support plan

[0806] 1. User: Submits a request for a post-disaster recovery plan.

[0807] 2. Server: The system analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, equipment) using optimization models and simulation tools.

[0808] 3. Server: Generates the optimal reconstruction assistance plan and presents it to the user.

[0809] 4. Device: The user checks the reconstruction assistance plan in a VR environment.

[0810] Integrating virtual cities with the real world

[0811] Real-World Data Collection

[0812] 1. User: Enters real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[0813] 2. Terminal: Sends the collected information to the server.

[0814] 3. Server: Integrates real-world data into the virtual city database and centrally manages virtual city and real-world data.

[0815] Providing integrated data and updating urban planning

[0816] 1. Server: Updates real-world urban planning and disaster response measures based on the integrated data.

[0817] 2. Device: Provide users with up-to-date planning information and gather feedback.

[0818] 3. User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[0819] Specific examples

[0820] Users upload landscape images taken with their smartphones to the system through a dedicated application. The server classifies the images and generates a prototype of the virtual city using Unity. The user then inputs further detailed requirements, and the server generates a detailed 3D model based on those requirements. The user then sets up an earthquake scenario in the VR environment, and the server runs the earthquake simulation and provides evacuation routes and support points.

[0821] Prompt Sentence Examples

[0822] "Please explain in detail the process by which a user uploads landscape images they have taken, and the system classifies them to generate a prototype of a virtual city. Also, please explain with concrete examples the steps by which a user sets up an earthquake scenario in a VR environment and checks the results."

[0823] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0824] Step 1:

[0825] Users take images of urban or natural scenery using a smartphone or digital camera and upload them to the system through a dedicated application. In this case, the input is the captured image data, and the output is the uploaded image data.

[0826] Step 2:

[0827] The terminal sends the scenery image uploaded by the user to the server. At this time, the input is the scenery image data present on the user's terminal, and the output is the image data sent to the server.

[0828] Step 3:

[0829] The server stores the received landscape images in a database and automatically tags and classifies them. Specifically, it uses an image recognition algorithm to assign tags such as "mountain," "river," and "building." The input is the landscape image data sent to the server, and the output is the tagged image data.

[0830] Step 4:

[0831] The server uses the stored landscape images to train an image recognition model (e.g., a convolutional neural network). Specifically, it uses machine learning libraries such as TensorFlow and PyTorch. In this case, the input is tagged landscape image data, and the output is the trained image recognition model.

[0832] Step 5:

[0833] The server uses a trained image recognition model to generate prototypes of terrain and infrastructure from landscape image data. Specifically, it uses game engines such as Unity or Unreal Engine. The input is the trained image recognition model and landscape image data, and the output is a prototype of a virtual city.

[0834] Step 6:

[0835] The server reflects earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.) in the generated prototype. At this time, the input is the virtual city prototype and the design standards, and the output is a prototype to which the earthquake-resistant standards have been applied.

[0836] Step 7:

[0837] Users input requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system using a dedicated application or a web interface. At this time, the input is the user's requirement data, and the output is the requirement data sent to the system.

[0838] Step 8:

[0839] The terminal transmits the user's input contents to the server in real time, where the input is the user's requirement data and the output is the requirement data transmitted to the server.

[0840] Step 9:

[0841] The server analyzes the user requirements, incorporates them into the prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment. The input is the user requirement data sent to the server, and the output is the detailed 3D model.

[0842] Step 10:

[0843] The user sets a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment. At this time, the input is the user's earthquake scenario setting data, and the output is the set scenario data.

[0844] Step 11:

[0845] The terminal transmits scenario setting information to the server. At this time, the input is the earthquake disaster scenario setting data of the user, and the output is the scenario data transmitted to the server.

[0846] Step 12:

[0847] The server runs the simulation in a virtual city environment and provides evacuation routes and support points. The input is the transmitted scenario data and a detailed 3D model, and the output is the simulation results.

[0848] Step 13:

[0849] A user sends a request for a post-disaster reconstruction assistance plan, where the input is the request data for the reconstruction assistance plan and the output is the request data sent to the server.

[0850] Step 14:

[0851] The server analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, and equipment) using optimization models and simulation tools. The inputs are simulation results and resource data, and the output is an optimal reconstruction assistance plan.

[0852] Step 15:

[0853] The server generates a reconstruction assistance plan and presents it to the user. At this time, the input is the optimal reconstruction assistance plan, and the output is the reconstruction assistance plan presented to the user.

[0854] Step 16:

[0855] The user checks the reconstruction support plan in the VR environment. The input is the reconstruction support plan presented to the user, and the output is the user's confirmation result.

[0856] Step 17:

[0857] Users input real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application, where the input is the real-world city information and the output is the city information entered into the application.

[0858] Step 18:

[0859] The terminal sends the collected information to the server, where the input is the city information entered into the application and the output is the city information sent to the server.

[0860] Step 19:

[0861] The server integrates real-world data into the virtual city database and manages the virtual city and real-world data in a unified manner. At this time, the input is the real-world city information sent to the server, and the output is the integrated database.

[0862] Step 20:

[0863] The server updates the actual city plans and disaster response measures based on the integrated data. At this time, the input is the integrated database, and the output is the updated city plans and disaster response measures.

[0864] Step 21:

[0865] The terminal provides users with up-to-date planning information and collects their feedback, where the input is updated urban planning and disaster response measures, and the output is user feedback.

[0866] Step 22:

[0867] Users can submit their opinions and make necessary corrections or suggestions based on the information provided. The input is the updated urban plan, disaster response measures, and the user's opinions, and the output is feedback data.

[0868] (Application example 1)

[0869] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0870] Modern factories and urban infrastructure are required to be more resilient to natural disasters such as earthquakes. In particular, factories with large-scale facilities and complex infrastructure layouts face the challenge of minimizing damage in the event of an earthquake while maintaining an efficient and safe layout. It is also important to develop a rapid and effective post-earthquake reconstruction support plan. To address these challenges, it is necessary to conduct simulations in virtual space using images of real-world factory interiors.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0872] In this invention, the server includes means for collecting, tagging, and classifying landscape images, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and incorporating them into the virtual city prototype to generate a detailed 3D model, means for setting earthquake scenarios and running earthquake simulations in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, means for updating real-world city plans and earthquake response measures based on the integrated data, means for collecting landscape images of real-world factories and simulating the arrangement of equipment and layout in a virtual space, and means for proposing optimal arrangements and layout changes to minimize the impact of earthquakes. This enables the proposal of safe and efficient layouts using landscape images of real factories.

[0873] "Landscape images" are images of various urban and natural landscapes.

[0874] "Tagging" refers to assigning labels such as "mountain," "river," and "building" to landscape images.

[0875] "Classification" refers to sorting tagged landscape images into their respective categories.

[0876] An "image recognition model" is an algorithm for analyzing landscape images and identifying their content.

[0877] A "virtual city" is a model of a city created on a computer based on landscape images and user requirements.

[0878] The "prototype" is the first prototype of the virtual city.

[0879] A "detailed 3D model" is a specific and precise three-dimensional virtual city model that reflects the user's requirements.

[0880] An "earthquake disaster scenario" sets out specific earthquake conditions, such as the epicenter, seismic intensity, and time period.

[0881] An "earthquake simulation" is a simulation based on an earthquake scenario set in a virtual city.

[0882] An "evacuation route" is a route for safe evacuation in the event of an earthquake.

[0883] A "support point" is a base set up to carry out support activities in the event of a disaster.

[0884] The "Reconstruction Support Plan" is a plan for urban reconstruction after the earthquake.

[0885] "Necessary resources" refers to the building materials, personnel, equipment, etc. required to implement the reconstruction assistance plan.

[0886] "City information" refers to data about real cities, such as ongoing construction projects and the state of existing infrastructure.

[0887] "Integration" means incorporating real city information into a virtual city database and managing it centrally.

[0888] "Scenery images inside the factory" refer to images taken of each area inside the factory.

[0889] "Facility and layout arrangement" refers to the arrangement of machines, aisles, work areas, etc. within the factory.

[0890] "Optimal location" means arranging the equipment and layout within the factory in the most effective way to minimize the impact of an earthquake.

[0891] The system configuration and operation are specifically described below as an embodiment of the invention. This system uses landscape images of the factory to propose safe and efficient layouts, and supports earthquake disaster response and reconstruction efforts.

[0892] System Configuration

[0893] The hardware and software required for this system are as follows:

[0894] Hardware

[0895] Factory robot with a mobile camera

[0896] High-resolution camera

[0897] Devices installed in actual factories (PCs, tablets, etc.)

[0898] VR headset (e.g. HTC Vive, Oculus Rift, etc.)

[0899] software

[0900] Image processing library (OpenCV)

[0901] Image recognition model (CNN using Keras)

[0902] 3D model generation library (Trimesh)

[0903] Physics simulation library (PyBullet)

[0904] Detailed explanation of operation

[0905] Collecting landscape images and learning models

[0906] 1. Collection of landscape images

[0907] The factory robot moves around the factory, taking pictures of each area with a high-resolution camera and sending them to a terminal. This image data is then stored on a server, where it is tagged and classified.

[0908] 2. Training the image recognition model

[0909] The server uses the collected landscape images to train a convolutional neural network (CNN) and generate a model for identifying landscape images. This trained model is then used to analyze the layout and equipment within the factory.

[0910] Virtual Factory Construction and Simulation

[0911] 3. Virtual Factory Prototype Generation

[0912] Using a trained image recognition model, a prototype of the factory's terrain and infrastructure is generated from landscape image data, and this prototype is then adapted to reflect design standards that take earthquake resistance into account.

[0913] 4. Gathering and elaborating user requirements

[0914] Users input their requirements for the factory layout and equipment placement into the system via a terminal. These requirements are analyzed by the server and incorporated into the prototype. Finally, a detailed 3D model is generated and exported to the VR environment.

[0915] 5. Earthquake response simulation and reconstruction support

[0916] Users can set up specific earthquake scenarios in the VR environment. The server then runs a simulation based on this scenario, providing evacuation routes and support points. It also analyzes post-earthquake reconstruction support plans, calculates the necessary resources, and presents an optimal reconstruction plan.

[0917] 6. Real-World Data Integration

[0918] Users input real-world information about the factory through terminals, and this data is integrated into a virtual factory database on the server. Based on this integrated data, the real-world factory layout and earthquake response measures are updated as needed.

[0919] Examples of specific examples and prompts

[0920] For example, a prototype is generated by having a factory robot take pictures of the factory interior and uploading the collected images to a server, after which users can use a VR headset to set up earthquake scenarios and check the simulation results.

[0921] An example of a prompt to be sent to a generative AI model is as follows:

[0922] plaintext

[0923] "Please recognize the locations of machines and aisles in the factory from this landscape image and generate a 3D model."

[0924] This enables the analysis of landscape images and the efficient generation and simulation of virtual factories.

[0925] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0926] Step 1:

[0927] The user operates a factory robot and takes pictures of the factory interior using a high-resolution camera.

[0928] Input: Real-world environment inside a factory

[0929] Output: Landscape image data

[0930] Specific operation: The user activates the factory robot and moves it around a designated area. During the movement, the camera continuously captures images of the scenery and stores them in the robot's internal storage.

[0931] Step 2:

[0932] The terminal receives landscape image data from the factory robot and uploads it to the server.

[0933] Input: Scenery image data sent from a factory robot

[0934] Output: Landscape images stored on the server

[0935] Specific operation: The terminal connects to the factory robot via the network and receives the captured image data. The received data is checked on the terminal and, if there are no problems, is uploaded to the server.

[0936] Step 3:

[0937] The server classifies and tags the uploaded landscape images.

[0938] Input: Uploaded landscape image data

[0939] Output: Tagged landscape image data

[0940] How it works: The server uses image recognition algorithms to analyze each image, detecting features in the image and automatically assigning tags such as "machine," "aisle," and "work area." The classified image data is then stored in a database on the server.

[0941] Step 4:

[0942] The server uses the landscape image data to train a convolutional neural network (CNN) model.

[0943] Input: tagged landscape image data

[0944] Output: A trained CNN model

[0945] Specific operation: The server trains the CNN model using landscape images in the database as learning data. Once the model is trained, it is saved after checking its image recognition accuracy.

[0946] Step 5:

[0947] Generate virtual factory prototypes using trained image recognition models.

[0948] Input: trained CNN model, landscape image data

[0949] Output: 3D prototype model of the virtual factory

[0950] How it works: The server inputs landscape image data into a CNN model to extract information about equipment and layout. Based on the extracted information, a virtual factory prototype is generated using 3D modeling software.

[0951] Step 6:

[0952] The user inputs requirements regarding the layout and equipment placement within the factory via a terminal.

[0953] Input: User requirements for layout and equipment placement

[0954] Output: Detailed layout requirements data

[0955] Specific operation: The user uses the interface on the terminal to input the desired equipment layout, aisle locations, etc. This requirement data is sent to the server in real time.

[0956] Step 7:

[0957] The server reflects the user's requirements and generates a detailed 3D model.

[0958] Input: User requirement data, 3D prototype model of virtual factory

[0959] Output: Detailed 3D model

[0960] How it works: The server analyzes the user's requirements data and incorporates them into a prototype model of the virtual factory. A 3D model with a detailed layout is generated and exported to the VR environment.

[0961] Step 8:

[0962] Users can set up specific earthquake scenarios in a VR environment and run simulations.

[0963] Input: Earthquake scenario (epicenter, seismic intensity, time period, etc.)

[0964] Output: Earthquake simulation results (evacuation routes, affected areas, etc.)

[0965] Specific operation: The user puts on a VR headset and uses the interface to set up an earthquake scenario. The server runs a simulation based on the set scenario, and the results are displayed as a VR scenario.

[0966] Step 9:

[0967] The server generates a post-earthquake reconstruction support plan and calculates the required resources.

[0968] Input: Earthquake simulation results

[0969] Output: Recovery aid plan, list of necessary resources

[0970] Specific operation: The server analyzes the simulation results and creates a reconstruction assistance plan. It calculates the necessary resources such as building materials, personnel, and equipment, and proposes an optimal reconstruction plan based on that.

[0971] Step 10:

[0972] Users collect real-world factory information via their terminals and send it to the server.

[0973] Input: Real-world factory information (ongoing construction projects, infrastructure status, etc.)

[0974] Output: Consolidated factory data

[0975] How it works: Users input data about ongoing construction projects and existing infrastructure status into their terminals and send it to the server, where it is integrated into the virtual factory database.

[0976] Step 11:

[0977] The server updates the actual factory layout and earthquake response measures based on the integrated data.

[0978] Input: Integrated factory data

[0979] Output: Updated factory layout and earthquake response plan

[0980] Specific operation: The server analyzes the integrated data and updates the latest factory layout and earthquake response measures. The updated contents are notified to the user via the terminal, and corrections and suggestions are made as necessary.

[0981] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0982] This invention relates to a system that uses landscape images to build a virtual city and provide disaster response and reconstruction support for earthquakes and other disasters, and also combines it with an emotion engine that recognizes the user's emotions. This system analyzes the user's emotions and optimizes the design of the virtual city and reconstruction plans based on those analyses.

[0983] 1. Collecting landscape images and training the model

[0984] Collection of landscape images

[0985] User: Uploads images of urban and natural landscapes that he has taken to the system.

[0986] Terminal: The user's terminal sends the uploaded landscape image to the server.

[0987] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[0988] Image recognition model training

[0989] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[0990] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[0991] 2. Building a Virtual City

[0992] Virtual city prototyping

[0993] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[0994] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[0995] Gathering and elaborating user requirements

[0996] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[0997] Terminal: User input is sent to the server in real time.

[0998] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[0999] Generate detailed 3D models

[1000] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[1001] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[1002] 3. Earthquake response simulation and reconstruction support

[1003] Earthquake scenario setting and simulation

[1004] User: Set up a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1005] Terminal: Sends the configured earthquake scenario to the server.

[1006] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[1007] Server: As a result of the simulation, it calculates the locations of evacuation routes and important support facilities and generates corresponding maps.

[1008] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1009] Creation of a recovery support plan

[1010] User: Requests the generation of a post-disaster recovery plan.

[1011] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[1012] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1013] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[1014] 4. Incorporating an Emotional Engine

[1015] emotion recognition

[1016] Users: Express emotions through facial expressions, voice, and text input while using the system.

[1017] Device: Captures the user's facial expression data with a camera, records voice data with a microphone, and collects text input data.

[1018] Server: Analyzes the data sent from the device and uses an emotion engine to recognize the user's emotions, such as relief, anxiety, satisfaction, etc.

[1019] Utilizing Emotional Data

[1020] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels a lot of anxiety, it provides a design and support plan that emphasizes safety.

[1021] On the device: Present the optimized design or plan to the user and collect feedback again.

[1022] 5. Integrating virtual cities with the real world

[1023] Real-World Data Collection

[1024] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[1025] Terminal: Sends collected information to the server.

[1026] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1027] Providing integrated data and updating urban planning

[1028] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1029] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[1030] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[1031] For example, a user uploads landscape images they have taken, and the system identifies them to generate a prototype of a virtual city. The user then sets up an earthquake scenario in the VR environment, runs a simulation, and checks evacuation routes and support points. During this process, the system analyzes the user's emotional data and provides an optimized earthquake response plan based on their sense of security or anxiety.

[1032] The processing flow will be explained below.

[1033] Step 1:

[1034] User: Uploads landscape images to the system.

[1035] Terminal: Sends the uploaded landscape image to the server.

[1036] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[1037] Step 2:

[1038] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[1039] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[1040] Step 3:

[1041] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[1042] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[1043] Step 4:

[1044] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[1045] Terminal: User input is sent to the server in real time.

[1046] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[1047] Step 5:

[1048] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[1049] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[1050] Step 6:

[1051] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1052] Terminal: Sends the configured earthquake scenario to the server.

[1053] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[1054] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[1055] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1056] Step 7:

[1057] User: Requests the generation of a post-disaster recovery plan.

[1058] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[1059] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1060] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[1061] Step 8:

[1062] Users: Express emotions naturally while using the system. Express emotions while interacting with and operating the system through facial expressions, voice, text input, etc.

[1063] Device: Captures the user's facial expressions with a camera, records their voice with a microphone, and collects the text they type. This data is sent to the server in real time.

[1064] Step 9:

[1065] Server: Analyzes the data sent from the device and uses an emotion engine to recognize the user's emotions, such as relief, anxiety, satisfaction, etc.

[1066] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. If the user feels a lot of anxiety, the server will adjust the design and reconstruction plan to emphasize safety.

[1067] Step 10:

[1068] Server: Presents the optimized design or plan to the user and again collects feedback.

[1069] Device: Displays updated designs and plans to the user in the VR environment and sends feedback to the server.

[1070] User: Review the optimized designs and plans provided and provide any necessary feedback.

[1071] Step 11:

[1072] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[1073] Terminal: Sends collected information to the server.

[1074] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1075] Step 12:

[1076] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1077] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[1078] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[1079] The above are the specific program processing steps in the system that combines the emotion engine. This invention makes it possible to design virtual cities and support reconstruction efforts that take into account the emotions of users.

[1080] Example 2

[1081] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1082] In modern society, disaster response and reconstruction support are important, especially in urban environments. However, conventional systems have had difficulty integrating real-world city data with virtual city data and providing optimal plans that take user emotions into account. There is also a need for an integrated platform that can simulate specific disaster scenarios in a virtual reality environment and generate optimal response measures while performing real-time emotion analysis.

[1083] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1084] In this invention, the server includes means for collecting, tagging, and classifying landscape images, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan, means for analyzing user emotions and optimizing the virtual city design and reconstruction plan based on the recognized emotion data, means for collecting real city information and integrating it into a virtual city database, and means for updating the real city plan and earthquake response measures based on the integrated data. This enables the system to integrate real city data and virtual city data in real time and generate optimal disaster response and reconstruction plans that take user emotions into consideration.

[1085] A "landscape image" is a digital image of a city, nature, or other landscape.

[1086] "Tagging" is the process of assigning labels to images that indicate their characteristics, such as "mountain," "river," or "building."

[1087] "Classification" is the process of separating tagged images into different categories.

[1088] An "image recognition model" is a machine learning model that analyzes patterns in landscape images and automatically identifies their features.

[1089] "Virtual City Prototype" is the creation of an early stage city model in virtual space based on collected landscape images.

[1090] "Detailed 3D model" refers to a detailed three-dimensional city model that includes specific design information based on user requirements.

[1091] An "earthquake disaster scenario" is a hypothetical scenario that simulates earthquakes and their associated impacts that may occur under specific conditions.

[1092] "Earthquake simulation" is a simulation that applies an earthquake scenario to a virtual urban environment to analyze evacuation routes and support points in the event of an earthquake.

[1093] A "support point" is the location of a base where important support activities are carried out in the event of a disaster.

[1094] A "reconstruction support plan" is a detailed plan that includes the resources needed to rebuild a city after a disaster and a plan to rebuild infrastructure.

[1095] An "emotion engine" is software that analyzes emotions from a user's facial expressions, voice, and text input.

[1096] A "virtual reality environment" is a digital environment that allows users to experience an immersive virtual space.

[1097] "City information" refers to data about real cities that shows the state of buildings under construction and existing infrastructure.

[1098] A "database" is a system for systematically storing and managing landscape images, virtual city models, emotional data, etc.

[1099] "Integration" refers to combining multiple different data sets in order to manage and use them in a centralized manner.

[1100] This invention relates to a system that uses landscape images to build a virtual city and provide disaster response and reconstruction support, as well as a system that combines an emotion engine that recognizes the user's emotions. The system analyzes the user's emotions and optimizes the design of the virtual city and reconstruction plans based on those analyses.

[1101] 1. Collecting landscape images and training the model

[1102] Collection of landscape images

[1103] Users: Users upload images of urban and natural landscapes taken with their smartphones or digital cameras, thereby collecting data on realistic urban environments.

[1104] Terminal: The user's terminal sends these landscape images to the server.

[1105] Server: The server stores the received landscape images in a database and tags and classifies them. Specific tags include "mountain," "river," and "building." This process uses image recognition services such as Google Cloud Vision API.

[1106] Image recognition model training

[1107] Server: The server uses landscape images stored in a database to train an image recognition model. Specific software used includes TensorFlow and Keras.

[1108] Server: As a result of the training, a model that can accurately identify landscape images is generated. This model is saved in a database and used in subsequent processing.

[1109] 2. Building a Virtual City

[1110] Virtual city prototyping

[1111] Server: Utilizing a trained image recognition model, the server generates a prototype of a virtual city by analyzing landscape image data stored in a database. It uses 3D modeling software such as Unity.

[1112] Server: Apply earthquake-resistant design standards, such as seismic strength and evacuation routes, to the prototype and make adjustments.

[1113] Gathering and elaborating user requirements

[1114] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, setting of residential areas) through a web form.

[1115] Terminal: Sends user-entered data to the server in real time.

[1116] Server: Analyzes the received requirements and incorporates them into the virtual city prototype.

[1117] Generate detailed 3D models

[1118] Server: Generate a detailed 3D model of the virtual city using 3D modeling tools such as Blender or Maya.

[1119] Server: The generated 3D models are stored in a database and exported to a VR environment (e.g., Oculus Rift compatible).

[1120] 3. Earthquake response simulation and reconstruction support

[1121] Earthquake scenario setting and simulation

[1122] User: Set up a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1123] Device: Sends settings to the server via the VR headset.

[1124] Server: Using simulation tools such as Unity, we run earthquake simulations in a virtual city environment, and analyze the results to determine evacuation routes and the locations of important support points.

[1125] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1126] Creation of a recovery support plan

[1127] User: Requests a post-disaster recovery plan from the system.

[1128] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.). This utilizes Unity's analysis functions.

[1129] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1130] Device: After viewing the reconstruction support plan in the VR environment, the user sends specific feedback to the server.

[1131] 4. Incorporating an Emotional Engine

[1132] emotion recognition

[1133] User: Expresses emotions through facial expressions, voice, and text input while using the system.

[1134] Device: Captures facial expressions with a camera, records audio with a microphone, and collects text input data.

[1135] Server: Analyzes the collected data using an emotion recognition API (e.g., Microsoft Azure Emotion API) to determine the user's emotions. For example, it identifies "relief," "anxiety," "satisfaction," etc.

[1136] Utilizing Emotional Data

[1137] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels anxious, it provides a design with enhanced safety and a support plan.

[1138] On the device: Present the optimized design or plan to the user and collect feedback again.

[1139] 5. Integrating virtual cities with the real world

[1140] Real-World Data Collection

[1141] Users: Enter real-world city information through the application, including the status of buildings under construction and the state of existing infrastructure.

[1142] Terminal: Sends collected information to the server.

[1143] Server: Integrates real-world data into the virtual city database and centrally manages data on the virtual city and real city.

[1144] Providing integrated data and updating urban planning

[1145] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1146] Device: Provides updated information to users, notifying them of the latest urban planning information and earthquake response measures.

[1147] User: Checks the provided information, sends necessary feedback to the server, and considers specific measures and actions based on this information.

[1148] Examples of specific examples and prompts

[1149] For example, a user uploads landscape images they have taken, and the system identifies them to generate a prototype of a virtual city. The user then sets up an earthquake scenario in the VR environment and runs a simulation to check evacuation routes and support points. During this process, the system analyzes the user's emotional data and provides an optimized earthquake response plan based on their sense of security or anxiety.

[1150] Example prompt sentence:

[1151] Please upload this landscape image.

[1152] "Please enter the requirements for your virtual city."

[1153] "Set up an earthquake scenario."

[1154] "Use your camera and microphone to express your emotions."

[1155] "Please enter real city information."

[1156] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1157] Step 1: Collecting landscape images

[1158] Users: Users upload images of city and natural scenery taken with their smartphones or digital cameras.

[1159] Input: Landscape image file (e.g. JPEG, PNG format)

[1160] Terminal: The user's terminal sends the uploaded landscape image to the server.

[1161] Server: The server stores the received landscape images in a database. It also uses a Python script to automatically tag and classify the images as "mountains," "rivers," "buildings," etc. This process uses the Google Cloud Vision API.

[1162] Output: A tagged and classified landscape image database

[1163] How it works: Users provide cityscape images following the prompt "Please upload," and the server stores and automatically tags them.

[1164] Step 2: Training the image recognition model

[1165] Server: The server uses the landscape images stored in the database to train an image recognition model using TensorFlow and Keras.

[1166] Input: A tagged and classified landscape image database

[1167] Data processing: Extract features from image data and train a model using machine learning algorithms.

[1168] Output: A trained, highly accurate image recognition model

[1169] How it works: The server retrieves landscape images from the database and trains them with the TensorFlow library to generate an image recognition model.

[1170] Step 3: Prototype the virtual city

[1171] Server: The server uses a trained image recognition model to analyze landscape image data from the database and generate a prototype of the virtual city. It uses 3D modeling software such as Unity.

[1172] Input: trained image recognition model, landscape image database

[1173] Data processing: Based on landscape images, prototypes are generated using 3D modeling software and seismic design criteria are applied.

[1174] Output: A prototype of a seismically designed virtual city

[1175] How it works: The server recognizes buildings and natural elements from landscape images and builds an initial virtual city model using Unity or Blender.

[1176] Step 4: Gathering and elaborating user requirements

[1177] User: The user fills out a web form with the requirements for the virtual city, including the placement of public facilities, the design of the transport infrastructure, and the setting of residential areas.

[1178] Input: User-entered requirement data (web form)

[1179] Terminal: The user terminal sends inputs to the server in real time.

[1180] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[1181] Output: Prototype reflecting user requirements

[1182] Specific operation: The user enters data into a web form through a prompt that says, "Please enter the requirements for your virtual city." The server analyzes this data and reflects it in the prototype.

[1183] Step 5: Generate a detailed 3D model

[1184] Server: The server uses 3D modeling tools such as Blender or Maya to generate a detailed 3D model of the virtual city based on the user requirements.

[1185] Input: Prototype reflecting user requirements

[1186] Data manipulation: Use 3D tools to refine and perfect your prototype.

[1187] Output: A detailed 3D model of a virtual city

[1188] What it does: The server takes requirements from the user and generates a detailed 3D model in Blender or Maya.

[1189] Step 6: Setting up and simulating earthquake scenarios

[1190] User: Set a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1191] Input: Earthquake scenario (VR environment)

[1192] Device: Sends settings to the server via the VR headset.

[1193] Server: Using simulation tools such as Unity, we run earthquake simulations in a virtual city environment and analyze the results.

[1194] Data processing: Analyze the simulation results and calculate the locations of evacuation routes and support points.

[1195] Output: Simulation results including evacuation routes and assistance points

[1196] Specific operation: The user puts on the VR headset and sets up the scenario according to the prompt, "Please set up the earthquake scenario." The server receives the data and runs the simulation.

[1197] Step 7: Generate a recovery plan

[1198] User: The user requests the system to generate a post-disaster recovery plan.

[1199] Input: Request (instructions to the system)

[1200] Server: Analyzes the impact of the earthquake based on the simulation results and calculates the required resources. Utilizing Unity's analysis functions.

[1201] Data processing: Calculate the necessary resources (building materials, personnel, equipment, etc.) and generate an efficient reconstruction support plan.

[1202] Output: Recovery assistance plan, infrastructure reconstruction plan, housing reconstruction plan

[1203] Specific operation: The user requests, "Please generate a post-disaster reconstruction support plan," and the server calculates the necessary resources and generates the support plan based on this.

[1204] Step 8: Emotion Recognition

[1205] User: The user expresses their emotions through facial expressions, voice, and text input while using the system.

[1206] Input: facial expressions, voice, text data

[1207] Device: Uses a camera and microphone to capture facial expressions and voice, and collects text input data.

[1208] Server: Analyze user emotions from collected data using an emotion recognition API (e.g., Microsoft Azure Emotion API).

[1209] Data processing: Facial expressions, voice, and text data are analyzed using an emotion engine to identify emotions (e.g., relief, anxiety).

[1210] Output: Recognized emotion data

[1211] Specific behavior: While using the system, the user follows the prompt: "Please use your camera and microphone to express your emotions." The server analyzes the captured data and identifies the emotions.

[1212] Step 9: Leverage sentiment data

[1213] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels anxious, it provides a design and support plan that enhances safety.

[1214] Input: Recognized emotion data, virtual city data

[1215] Data processing: Adjust and optimize designs and plans based on sentiment data.

[1216] Output: Optimized virtual city design and reconstruction support plan

[1217] Specific operation: The server takes in the emotional data, makes design changes, for example to increase the sense of security, and shows them to the user.

[1218] Step 10: Real-world data collection and integration

[1219] User: Uses the application to input real-world city information (status of buildings under construction, existing infrastructure, etc.).

[1220] Input: Real city information

[1221] Terminal: Sends collected information to the server.

[1222] Server: Integrates real-world data into the virtual city database and centrally manages real and virtual data.

[1223] Output: Integrated city database

[1224] Specific operation: The user inputs real city information according to the application's prompts, and the server receives it and integrates it with the virtual city data.

[1225] Step 11: Providing integrated data and updating urban plans

[1226] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1227] Input: Integrated City Database

[1228] Data processing: Analyze the integrated data and update real-world urban planning and disaster response measures.

[1229] Output: Updated urban planning information, earthquake response measures

[1230] Specific operation: The server analyzes the integrated data, generates the latest urban planning information and earthquake response measures, and notifies the user.

[1231] The specific actions at each step ensure that the entire system functions efficiently, enabling users to obtain optimal disaster response and recovery plans in real time.

[1232] (Application example 2)

[1233] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1234] Although there are systems that build virtual cities based on real-world landscape images and provide disaster response and reconstruction support, conventional systems are not optimized to take user emotions into account, making it difficult to provide designs and plans that provide the most peace of mind and satisfaction to users. Furthermore, virtual store designs are not optimized based on user emotions, making it difficult to provide the optimal shopping experience tailored to each individual user.

[1235] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting landscape images and tagging and classifying them, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model, means for setting an earthquake scenario and performing an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating required resources, and presenting the reconstruction plan, means for collecting real city information and integrating it into a virtual city database, means for updating real city plans and earthquake response measures based on the integrated data, means for analyzing user emotion data using an emotion recognition engine and optimizing the virtual city design and reconstruction plan based on the emotion data, and means for generating a virtual store based on landscape images uploaded by a user, adjusting the design and product placement of the virtual store based on the emotion data, and providing the user with the virtual store optimized in the virtual reality environment. This will enable disaster response and reconstruction support plans to be provided that take into account the user's emotions, and will also enable the creation of an optimal shopping experience in virtual stores.

[1236] "Landscape images" are photographs or video footage of natural or urban landscapes.

[1237] "Tagging" is the act of assigning keywords and classification information to data or images.

[1238] "Classification" is the act of separating data or information into specific categories or groups.

[1239] An "image recognition model" is a mathematical model that uses machine learning algorithms to analyze images and identify their content.

[1240] A "virtual city" is an urban environment generated by computer simulation.

[1241] A "prototype" is an early model or prototype of a new system or product.

[1242] "User requirements" refers to the functions and conditions that users desire for the system.

[1243] A "detailed 3D model" is a model of an object or environment that is recreated in detail in three-dimensional space.

[1244] An "earthquake disaster scenario" is a simulation scenario that assumes the occurrence of an earthquake or related disaster.

[1245] "Earthquake simulation" is a system that uses a computer to recreate earthquakes and related disasters and analyze their impact.

[1246] An "evacuation route" is a route that people can use to safely evacuate in the event of a disaster.

[1247] "Support points" are important locations where support activities are carried out during disasters.

[1248] A "reconstruction support plan" is a plan to support the reconstruction of a region after an earthquake or other disaster.

[1249] "Resources" are the resources and materials required to run a system or project.

[1250] "Real city information" refers to data and contextual information about real cities.

[1251] The "Virtual City Database" is a database system that manages and stores various data related to virtual cities.

[1252] An "emotion recognition engine" is a system that analyzes a user's facial expressions, voice, text input, etc. to identify their emotions.

[1253] A "virtual store" is a virtual store built on the Internet or in a virtual reality space.

[1254] "Design" is the act of designing objects or environments with shapes and structures that have specific purposes and functions.

[1255] "Product placement" refers to the act of deciding how to place products in a store.

[1256] A "virtual reality environment" is a three-dimensional space generated using computer technology in which a user can act or experience something.

[1257] This system uses landscape images to build a virtual city, recognizes the user's emotions, and provides optimized designs and plans based on those emotions. This system is targeted at disaster response and reconstruction support, as well as virtual store generation and optimization.

[1258] System configuration

[1259] This system uses the following hardware and software:

[1260] Hardware

[1261] Smartphone (camera and microphone for capturing landscape images and emotion recognition)

[1262] Smart glasses (optional, for displaying virtual reality environments and emotion recognition)

[1263] Server (data processing and storage)

[1264] software

[1265] Image recognition model (convolutional neural network)

[1266] Emotion recognition engine (machine learning model)

[1267] Virtual Reality (VR) Engine

[1268] System Operation

[1269] 1. Collecting and tagging landscape images

[1270] Users use their smartphones to take pictures of urban or natural landscapes and upload them to the system. The server receives the uploaded images and tags and classifies them, adding tags such as "mountain," "river," and "building."

[1271] 2. Training the image recognition model

[1272] The server trains an image recognition model using landscape images stored in a database, using a convolutional neural network (CNN) to generate a highly accurate model for identifying landscape images.

[1273] 3. Prototype generation of virtual city

[1274] Using a trained image recognition model, the system analyzes landscape image data to generate a prototype of a virtual city, taking into account earthquake resistance standards and evacuation routes.

[1275] 4. Reflecting user requirements and generating detailed 3D models

[1276] Users input the requirements for a virtual city into the system, such as the layout of public facilities, the design of transportation infrastructure, the setting of residential areas, etc. The server analyzes the received requirements, incorporates them into the virtual city prototype, and generates a detailed 3D model.

[1277] 5. Earthquake Disaster Simulation and Generation of Recovery Support Plans

[1278] Based on the earthquake scenario set by the user, the server runs an earthquake simulation in a virtual city environment. Based on the simulation results, it calculates evacuation routes and support points, and generates a reconstruction support plan. It also calculates the necessary resources (building materials, personnel, equipment, etc.).

[1279] 6. Emotion Recognition and Optimization

[1280] When using the system, the camera and microphone on the smartphone or smart glasses are used to collect the user's facial expressions and speech. An emotion recognition engine analyzes this data and identifies the user's feelings of relief, anxiety, satisfaction, etc. The server then uses the recognized emotion data to optimize the design of the virtual city and reconstruction plans.

[1281] 7. Virtual store generation and optimization

[1282] The server generates a virtual store based on landscape images uploaded by users. The design and product layout of the virtual store are adjusted based on emotional data from users while using the system. The optimized virtual store is provided to users in a virtual reality environment, where they can experience and shop.

[1283] Prompt Sentence Examples

[1284] "Log in to this virtual store app and upload a landscape image taken with your smartphone. A virtual store will be generated based on that image. Next, walk around the store freely and look at the products. Actively express emotions such as joy, excitement, or anxiety when you pick up a particular product. The emotion recognition engine analyzes these emotions and suggests optimal store design and product placement."

[1285] The above is an embodiment of the present invention. This system makes it possible to provide disaster response and reconstruction support plans that take into account the user's emotions, and also to realize an optimal shopping experience in a virtual store.

[1286] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1287] Step 1:

[1288] Collecting and tagging landscape images

[1289] Users use their smartphones to take images of urban or natural landscapes and upload them to the system. The server receives the uploaded images and tags and classifies them. The input is the landscape image taken by the user, and the output is the tagged and classified landscape image. Specifically, the server uses a convolutional neural network (CNN) to extract image features and assign tags such as "mountain," "river," and "building."

[1290] Step 2:

[1291] Image recognition model training

[1292] The server trains an image recognition model using landscape images stored in a database. The input is tagged and classified landscape images, and the output is a trained image recognition model. Specifically, the server trains a CNN using a large landscape image dataset, which generates a highly accurate model for accurately identifying landscape images.

[1293] Step 3:

[1294] Virtual city prototyping

[1295] The server uses a trained image recognition model to analyze input landscape image data and generate a prototype of the virtual city. The output is an initial prototype of the virtual city. Specifically, the server extracts features from the landscape image data, generates 3D models of buildings and terrain, and uses these to create the basic structure of the virtual city.

[1296] Step 4:

[1297] Reflecting user requirements and generating detailed 3D models

[1298] Users input their requirements for a virtual city into the system, including the layout of public facilities, the design of transportation infrastructure, and the establishment of residential areas. The server receives this information, analyzes it, and applies it to a prototype of the virtual city, generating a detailed 3D model. The input is the user's requirements, and the output is a 3D model based on the user's requirements. Specifically, the server adjusts the layout of the prototype based on the conditions entered by the user and runs an algorithm to generate a detailed model.

[1299] Step 5:

[1300] Earthquake disaster simulation and generation of reconstruction support plans

[1301] The server runs an earthquake simulation in a virtual city environment, using an earthquake scenario set by the user as input. The input is the earthquake scenario, and the output is evacuation routes and support points based on the simulation results. The server analyzes the simulation results, calculates the resources required to generate a post-earthquake reconstruction support plan, and presents the reconstruction plan. Specifically, it simulates the impact within the virtual city based on parameters such as the location, time, and seismic intensity of the earthquake, and identifies evacuation routes and key points.

[1302] Step 6:

[1303] Emotion Recognition and Optimization

[1304] When a user uses the system, the system collects their facial expressions and speech using a camera and microphone installed on their smartphone or smart glasses. The input is the user's facial expression and voice data, and the output is analyzed emotional data. The server analyzes this data using an emotion recognition engine to identify the user's feelings of relief, anxiety, satisfaction, etc. Furthermore, it optimizes the design and reconstruction plan of the virtual city based on the emotional data. Specifically, it runs an algorithm to apply a design and layout that makes the user feel at ease.

[1305] Step 7:

[1306] Virtual store generation and optimization

[1307] The server generates a virtual store based on landscape images uploaded by users. The input is landscape images taken by users and emotional data, and the output is an optimized virtual store design. The server adjusts the virtual store design and product layout based on the emotional data. The optimized virtual store is provided to users in a virtual reality environment, where they can experience and shop. Specifically, the system analyzes emotional data in real time as users browse products, and sequentially optimizes product layout and design.

[1308] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1309] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1310] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1311] [Third embodiment]

[1312] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1313] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1314] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1315] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1316] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1317] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1318] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1319] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1320] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1321] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1322] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1323] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1324] The following system configuration and operation will be described as an embodiment of the invention: This system uses landscape images to build a virtual city and to respond to earthquake disasters and provide reconstruction support.

[1325] 1. Collecting landscape images and training the model

[1326] Collection of landscape images

[1327] User: Uploads images of urban and natural landscapes that he has taken to the system.

[1328] Terminal: The user's terminal sends the uploaded landscape image to the server.

[1329] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[1330] Image recognition model training

[1331] Server: Uses the stored landscape images to train an image recognition model such as a convolutional neural network (CNN).

[1332] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[1333] 2. Building a Virtual City

[1334] Virtual city prototyping

[1335] Server: Uses trained image recognition models to generate terrain and infrastructure prototypes from landscape image data.

[1336] Server: This prototype will be designed to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[1337] Gathering and elaborating user requirements

[1338] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system.

[1339] Terminal: User input is sent to the server in real time.

[1340] Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements and exports it to the VR environment.

[1341] 3. Earthquake response simulation and reconstruction support

[1342] Setting up a disaster scenario

[1343] User: Set up a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[1344] Terminal: Sends scenario setting information to the server.

[1345] Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[1346] Creation of a recovery support plan

[1347] User: Submits a request for a post-disaster recovery plan.

[1348] Server: Analyzes the impact of the earthquake and calculates the necessary resources (e.g., building materials, personnel, and equipment). Generates an optimal reconstruction support plan and presents it to the user.

[1349] Device: The user checks the reconstruction assistance plan in a VR environment.

[1350] 4. Integrating virtual cities with the real world

[1351] Real-World Data Collection

[1352] Users: Enter real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[1353] Terminal: Sends collected information to the server.

[1354] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1355] Providing integrated data and updating urban planning

[1356] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1357] Device: Provide users with up-to-date planning information and gather feedback.

[1358] User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[1359] For example, a user can upload landscape images they have taken, and the system will analyze them and generate a prototype of a virtual city. After that, the user can set up an earthquake scenario in the VR environment and check the generated simulation results, which will enable efficient earthquake response and reconstruction support.

[1360] The processing flow will be explained below.

[1361] Step 1:

[1362] User: Uploads landscape images to the system.

[1363] Terminal: Sends the uploaded landscape image to the server.

[1364] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[1365] Step 2:

[1366] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[1367] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[1368] Step 3:

[1369] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[1370] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[1371] Step 4:

[1372] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[1373] Terminal: Sends the requirements entered by the user to the server in real time.

[1374] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[1375] Step 5:

[1376] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[1377] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[1378] Step 6:

[1379] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1380] Terminal: Sends the configured earthquake scenario to the server.

[1381] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[1382] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[1383] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1384] Step 7:

[1385] User: Requests the generation of a post-disaster recovery plan.

[1386] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[1387] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1388] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[1389] Step 8:

[1390] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[1391] Terminal: Sends collected information to the server.

[1392] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1393] Step 9:

[1394] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1395] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[1396] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[1397] The above are the processing steps of the specific program for carrying out the invention.

[1398] Example 1

[1399] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1400] Conventional urban planning and earthquake response systems have difficulty integrating and operating real-world data with virtual environments, resulting in a lack of integrated means for efficient urban planning, disaster simulation, and reconstruction support. Furthermore, it is difficult to update user-provided information in real time in the virtual city, making it difficult to effectively use these systems in situations where a rapid and accurate response is required.

[1401] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1402] In this invention, the server includes means for collecting landscape images taken by users and tagging and classifying them, means for training an image recognition model such as a convolutional neural network using the landscape image data, means for generating a virtual city prototype using the trained image recognition model, means for inputting user requirements, means for converting the virtual city prototype into a detailed 3D model reflecting the user requirements, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for using the server to generate a post-earthquake reconstruction support plan, calculate the required resources, and present the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, and means for updating real-world city plans and earthquake response measures based on the integrated data. This enables information provided by users to be quickly and accurately reflected in the virtual city, enabling efficient urban planning, earthquake response, and reconstruction support.

[1403] "User" refers to a person who uploads landscape images to the system, inputs requirements, and checks and uses the simulation results and reconstruction plans.

[1404] "Terminal" refers to a device that a user connects to and operates the system, and is used to upload landscape images, input requirements, and check simulation results.

[1405] The "server" refers to the central processing unit that is the back-end computing resource that supports the entire system and performs tasks such as collecting images, tagging, classifying, learning, generating prototypes, running simulations, generating reconstruction assistance plans, and integrating data.

[1406] "Landscape images" refer to image data of urban and natural landscapes that users take and upload to the system.

[1407] "Tagging" refers to the process of categorizing collected landscape images by assigning keywords such as "mountain," "river," and "building."

[1408] "Classification" refers to the process of organizing tagged landscape images into categories and storing them in a database.

[1409] An "image recognition model" refers to a machine learning model trained to identify and classify landscape images using a convolutional neural network (CNN) or similar.

[1410] "Virtual city" refers to a virtual urban environment created based on landscape image data, generated using a trained image recognition model.

[1411] "Prototype" refers to an early model or structure of a virtual city, a temporary mockup before being refined with further user requirements and data.

[1412] "User requirements" refers to the various specific requests and specifications for the virtual city that the user inputs into the system.

[1413] "Detailed 3D model" refers to a detailed three-dimensional virtual city model that is generated as a result of reflecting user requirements.

[1414] "Earthquake scenario" refers to the conditions for an earthquake to occur (e.g., epicenter, seismic intensity, time period) set in a virtual city environment.

[1415] "Earthquake simulation" refers to the process of simulating the impact and evacuation routes of an earthquake occurring within a virtual city environment based on a set earthquake scenario.

[1416] An "evacuation route" refers to a route within a virtual city that is set up for safe evacuation in an earthquake simulation.

[1417] "Support points" refer to points or locations where evacuees can receive support in earthquake simulations.

[1418] "Reconstruction Support Plan" refers to a plan that includes the resources (e.g., building materials, personnel, and equipment) needed to rebuild a virtual city after a disaster.

[1419] "Real-world urban information" refers to data such as the current state of cities and ongoing construction projects in the real world.

[1420] A "virtual city database" refers to a database for centrally managing real city information and virtual city information.

[1421] "Urban planning" refers to plans and policies for the future development and maintenance of virtual and real cities.

[1422] "Earthquake response measures" refers to plans that include actions, procedures, and preparations to be taken in the event of an earthquake.

[1423] This invention is a system for constructing a virtual city using landscape images and for responding to earthquake disasters and supporting reconstruction efforts. How each step is carried out will be described in detail below.

[1424] Collecting landscape images and learning models

[1425] Collection of landscape images

[1426] 1. User: Takes images of urban or natural scenery with a smartphone or digital camera and uploads them to the system using a dedicated application.

[1427] 2. Device: The user's device (e.g., smartphone or PC) sends the uploaded landscape images to the server.

[1428] 3. Server: The received landscape images are stored in a database and tagged and classified. Tags are automatically assigned using an image recognition algorithm, such as "mountain," "river," and "building." Classification is based on these tags.

[1429] Image recognition model training

[1430] 1. Server: Using the stored landscape images, an image recognition model such as a convolutional neural network (CNN) is trained using a machine learning library such as TensorFlow or PyTorch.

[1431] 2. Server: As a result of the training, a reliable model for identifying landscape images is generated and stored in a database.

[1432] Building a Virtual City

[1433] Virtual city prototyping

[1434] 1. Server: Using trained image recognition models, prototypes of terrain and infrastructure are generated from landscape image data. For example, prototypes are created using game engines such as Unity or Unreal Engine.

[1435] 2. Server: This prototype is applied with algorithms to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[1436] Gathering and elaborating user requirements

[1437] 1. User: Uses a dedicated application or web interface to input specific requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting up residential areas) into the system.

[1438] 2. Terminal: Sends user input to the server in real time.

[1439] 3. Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment.

[1440] Earthquake response simulation and reconstruction support

[1441] Setting up a disaster scenario

[1442] 1. User: Set a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[1443] 2. Terminal: Sends the configured scenario information to the server.

[1444] 3. Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[1445] Creation of a recovery support plan

[1446] 1. User: Submits a request for a post-disaster recovery plan.

[1447] 2. Server: The system analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, equipment) using optimization models and simulation tools.

[1448] 3. Server: Generates the optimal reconstruction assistance plan and presents it to the user.

[1449] 4. Device: The user checks the reconstruction assistance plan in a VR environment.

[1450] Integrating virtual cities with the real world

[1451] Real-World Data Collection

[1452] 1. User: Enters real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[1453] 2. Terminal: Sends the collected information to the server.

[1454] 3. Server: Integrates real-world data into the virtual city database and centrally manages virtual city and real-world data.

[1455] Providing integrated data and updating urban planning

[1456] 1. Server: Updates real-world urban planning and disaster response measures based on the integrated data.

[1457] 2. Device: Provide users with up-to-date planning information and gather feedback.

[1458] 3. User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[1459] Specific examples

[1460] Users upload landscape images taken with their smartphones to the system through a dedicated application. The server classifies the images and generates a prototype of the virtual city using Unity. The user then inputs further detailed requirements, and the server generates a detailed 3D model based on those requirements. The user then sets up an earthquake scenario in the VR environment, and the server runs the earthquake simulation and provides evacuation routes and support points.

[1461] Prompt Sentence Examples

[1462] "Please explain in detail the process by which a user uploads landscape images they have taken, and the system classifies them to generate a prototype of a virtual city. Also, please explain with concrete examples the steps by which a user sets up an earthquake scenario in a VR environment and checks the results."

[1463] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1464] Step 1:

[1465] Users take images of urban or natural scenery using a smartphone or digital camera and upload them to the system through a dedicated application. In this case, the input is the captured image data, and the output is the uploaded image data.

[1466] Step 2:

[1467] The terminal sends the scenery image uploaded by the user to the server. At this time, the input is the scenery image data present on the user's terminal, and the output is the image data sent to the server.

[1468] Step 3:

[1469] The server stores the received landscape images in a database and automatically tags and classifies them. Specifically, it uses an image recognition algorithm to assign tags such as "mountain," "river," and "building." The input is the landscape image data sent to the server, and the output is the tagged image data.

[1470] Step 4:

[1471] The server uses the stored landscape images to train an image recognition model (e.g., a convolutional neural network). Specifically, it uses machine learning libraries such as TensorFlow and PyTorch. In this case, the input is tagged landscape image data, and the output is the trained image recognition model.

[1472] Step 5:

[1473] The server uses a trained image recognition model to generate prototypes of terrain and infrastructure from landscape image data. Specifically, it uses game engines such as Unity or Unreal Engine. The input is the trained image recognition model and landscape image data, and the output is a prototype of a virtual city.

[1474] Step 6:

[1475] The server reflects earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.) in the generated prototype. At this time, the input is the virtual city prototype and the design standards, and the output is a prototype to which the earthquake-resistant standards have been applied.

[1476] Step 7:

[1477] Users input requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system using a dedicated application or a web interface. At this time, the input is the user's requirement data, and the output is the requirement data sent to the system.

[1478] Step 8:

[1479] The terminal transmits the user's input contents to the server in real time, where the input is the user's requirement data and the output is the requirement data transmitted to the server.

[1480] Step 9:

[1481] The server analyzes the user requirements, incorporates them into the prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment. The input is the user requirement data sent to the server, and the output is the detailed 3D model.

[1482] Step 10:

[1483] The user sets a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment. At this time, the input is the user's earthquake scenario setting data, and the output is the set scenario data.

[1484] Step 11:

[1485] The terminal transmits scenario setting information to the server. At this time, the input is the earthquake disaster scenario setting data of the user, and the output is the scenario data transmitted to the server.

[1486] Step 12:

[1487] The server runs the simulation in a virtual city environment and provides evacuation routes and support points. The input is the transmitted scenario data and a detailed 3D model, and the output is the simulation results.

[1488] Step 13:

[1489] A user sends a request for a post-disaster reconstruction assistance plan, where the input is the request data for the reconstruction assistance plan and the output is the request data sent to the server.

[1490] Step 14:

[1491] The server analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, and equipment) using optimization models and simulation tools. The inputs are simulation results and resource data, and the output is an optimal reconstruction assistance plan.

[1492] Step 15:

[1493] The server generates a reconstruction assistance plan and presents it to the user. At this time, the input is the optimal reconstruction assistance plan, and the output is the reconstruction assistance plan presented to the user.

[1494] Step 16:

[1495] The user checks the reconstruction support plan in the VR environment. The input is the reconstruction support plan presented to the user, and the output is the user's confirmation result.

[1496] Step 17:

[1497] Users input real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application, where the input is the real-world city information and the output is the city information entered into the application.

[1498] Step 18:

[1499] The terminal sends the collected information to the server, where the input is the city information entered into the application and the output is the city information sent to the server.

[1500] Step 19:

[1501] The server integrates real-world data into the virtual city database and manages the virtual city and real-world data in a unified manner. At this time, the input is the real-world city information sent to the server, and the output is the integrated database.

[1502] Step 20:

[1503] The server updates the actual city plans and disaster response measures based on the integrated data. At this time, the input is the integrated database, and the output is the updated city plans and disaster response measures.

[1504] Step 21:

[1505] The terminal provides users with up-to-date planning information and collects their feedback, where the input is updated urban planning and disaster response measures, and the output is user feedback.

[1506] Step 22:

[1507] Users can submit their opinions and make necessary corrections or suggestions based on the information provided. The input is the updated urban plan, disaster response measures, and the user's opinions, and the output is feedback data.

[1508] (Application example 1)

[1509] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1510] Modern factories and urban infrastructure are required to be more resilient to natural disasters such as earthquakes. In particular, factories with large-scale facilities and complex infrastructure layouts face the challenge of minimizing damage in the event of an earthquake while maintaining an efficient and safe layout. It is also important to develop a rapid and effective post-earthquake reconstruction support plan. To address these challenges, it is necessary to conduct simulations in virtual space using images of real-world factory interiors.

[1511] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1512] In this invention, the server includes means for collecting, tagging, and classifying landscape images, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and incorporating them into the virtual city prototype to generate a detailed 3D model, means for setting earthquake scenarios and running earthquake simulations in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, means for updating real-world city plans and earthquake response measures based on the integrated data, means for collecting landscape images of real-world factories and simulating the arrangement of equipment and layout in a virtual space, and means for proposing optimal arrangements and layout changes to minimize the impact of earthquakes. This enables the proposal of safe and efficient layouts using landscape images of real factories.

[1513] "Landscape images" are images of various urban and natural landscapes.

[1514] "Tagging" refers to assigning labels such as "mountain," "river," and "building" to landscape images.

[1515] "Classification" refers to sorting tagged landscape images into their respective categories.

[1516] An "image recognition model" is an algorithm for analyzing landscape images and identifying their content.

[1517] A "virtual city" is a model of a city created on a computer based on landscape images and user requirements.

[1518] The "prototype" is the first prototype of the virtual city.

[1519] A "detailed 3D model" is a specific and precise three-dimensional virtual city model that reflects the user's requirements.

[1520] An "earthquake disaster scenario" sets out specific earthquake conditions, such as the epicenter, seismic intensity, and time period.

[1521] An "earthquake simulation" is a simulation based on an earthquake scenario set in a virtual city.

[1522] An "evacuation route" is a route for safe evacuation in the event of an earthquake.

[1523] A "support point" is a base set up to carry out support activities in the event of a disaster.

[1524] The "Reconstruction Support Plan" is a plan for urban reconstruction after the earthquake.

[1525] "Necessary resources" refers to the building materials, personnel, equipment, etc. required to implement the reconstruction assistance plan.

[1526] "City information" refers to data about real cities, such as ongoing construction projects and the state of existing infrastructure.

[1527] "Integration" means incorporating real city information into a virtual city database and managing it centrally.

[1528] "Scenery images inside the factory" refer to images taken of each area inside the factory.

[1529] "Facility and layout arrangement" refers to the arrangement of machines, aisles, work areas, etc. within the factory.

[1530] "Optimal location" means arranging the equipment and layout within the factory in the most effective way to minimize the impact of an earthquake.

[1531] The system configuration and operation are specifically described below as an embodiment of the invention. This system uses landscape images of the factory to propose safe and efficient layouts, and supports earthquake disaster response and reconstruction efforts.

[1532] System Configuration

[1533] The hardware and software required for this system are as follows:

[1534] Hardware

[1535] Factory robot with a mobile camera

[1536] High-resolution camera

[1537] Devices installed in actual factories (PCs, tablets, etc.)

[1538] VR headset (e.g. HTC Vive, Oculus Rift, etc.)

[1539] software

[1540] Image processing library (OpenCV)

[1541] Image recognition model (CNN using Keras)

[1542] 3D model generation library (Trimesh)

[1543] Physics simulation library (PyBullet)

[1544] Detailed explanation of operation

[1545] Collecting landscape images and learning models

[1546] 1. Collection of landscape images

[1547] The factory robot moves around the factory, taking pictures of each area with a high-resolution camera and sending them to a terminal. This image data is then stored on a server, where it is tagged and classified.

[1548] 2. Training the image recognition model

[1549] The server uses the collected landscape images to train a convolutional neural network (CNN) and generate a model for identifying landscape images. This trained model is then used to analyze the layout and equipment within the factory.

[1550] Virtual Factory Construction and Simulation

[1551] 3. Virtual Factory Prototype Generation

[1552] Using a trained image recognition model, a prototype of the factory's terrain and infrastructure is generated from landscape image data, and this prototype is then adapted to reflect design standards that take earthquake resistance into account.

[1553] 4. Gathering and elaborating user requirements

[1554] Users input their requirements for the factory layout and equipment placement into the system via a terminal. These requirements are analyzed by the server and incorporated into the prototype. Finally, a detailed 3D model is generated and exported to the VR environment.

[1555] 5. Earthquake response simulation and reconstruction support

[1556] Users can set up specific earthquake scenarios in the VR environment. The server then runs a simulation based on this scenario, providing evacuation routes and support points. It also analyzes post-earthquake reconstruction support plans, calculates the necessary resources, and presents an optimal reconstruction plan.

[1557] 6. Real-World Data Integration

[1558] Users input real-world information about the factory through terminals, and this data is integrated into a virtual factory database on the server. Based on this integrated data, the real-world factory layout and earthquake response measures are updated as needed.

[1559] Examples of specific examples and prompts

[1560] For example, a prototype is generated by having a factory robot take pictures of the factory interior and uploading the collected images to a server, after which users can use a VR headset to set up earthquake scenarios and check the simulation results.

[1561] An example of a prompt to be sent to a generative AI model is as follows:

[1562] plaintext

[1563] "Please recognize the locations of machines and aisles in the factory from this landscape image and generate a 3D model."

[1564] This enables the analysis of landscape images and the efficient generation and simulation of virtual factories.

[1565] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1566] Step 1:

[1567] The user operates a factory robot and takes pictures of the factory interior using a high-resolution camera.

[1568] Input: Real-world environment inside a factory

[1569] Output: Landscape image data

[1570] Specific operation: The user activates the factory robot and moves it around a designated area. During the movement, the camera continuously captures images of the scenery and stores them in the robot's internal storage.

[1571] Step 2:

[1572] The terminal receives landscape image data from the factory robot and uploads it to the server.

[1573] Input: Scenery image data sent from a factory robot

[1574] Output: Landscape images stored on the server

[1575] Specific operation: The terminal connects to the factory robot via the network and receives the captured image data. The received data is checked on the terminal and, if there are no problems, is uploaded to the server.

[1576] Step 3:

[1577] The server classifies and tags the uploaded landscape images.

[1578] Input: Uploaded landscape image data

[1579] Output: Tagged landscape image data

[1580] How it works: The server uses image recognition algorithms to analyze each image, detecting features in the image and automatically assigning tags such as "machine," "aisle," and "work area." The classified image data is then stored in a database on the server.

[1581] Step 4:

[1582] The server uses the landscape image data to train a convolutional neural network (CNN) model.

[1583] Input: tagged landscape image data

[1584] Output: A trained CNN model

[1585] Specific operation: The server trains the CNN model using landscape images in the database as learning data. Once the model is trained, it is saved after checking its image recognition accuracy.

[1586] Step 5:

[1587] Generate virtual factory prototypes using trained image recognition models.

[1588] Input: trained CNN model, landscape image data

[1589] Output: 3D prototype model of the virtual factory

[1590] How it works: The server inputs landscape image data into a CNN model to extract information about equipment and layout. Based on the extracted information, a virtual factory prototype is generated using 3D modeling software.

[1591] Step 6:

[1592] The user inputs requirements regarding the layout and equipment placement within the factory via a terminal.

[1593] Input: User requirements for layout and equipment placement

[1594] Output: Detailed layout requirements data

[1595] Specific operation: The user uses the interface on the terminal to input the desired equipment layout, aisle locations, etc. This requirement data is sent to the server in real time.

[1596] Step 7:

[1597] The server reflects the user's requirements and generates a detailed 3D model.

[1598] Input: User requirement data, 3D prototype model of virtual factory

[1599] Output: Detailed 3D model

[1600] How it works: The server analyzes the user's requirements data and incorporates them into a prototype model of the virtual factory. A 3D model with a detailed layout is generated and exported to the VR environment.

[1601] Step 8:

[1602] Users can set up specific earthquake scenarios in a VR environment and run simulations.

[1603] Input: Earthquake scenario (epicenter, seismic intensity, time period, etc.)

[1604] Output: Earthquake simulation results (evacuation routes, affected areas, etc.)

[1605] Specific operation: The user puts on a VR headset and uses the interface to set up an earthquake scenario. The server runs a simulation based on the set scenario, and the results are displayed as a VR scenario.

[1606] Step 9:

[1607] The server generates a post-earthquake reconstruction support plan and calculates the required resources.

[1608] Input: Earthquake simulation results

[1609] Output: Recovery aid plan, list of necessary resources

[1610] Specific operation: The server analyzes the simulation results and creates a reconstruction assistance plan. It calculates the necessary resources such as building materials, personnel, and equipment, and proposes an optimal reconstruction plan based on that.

[1611] Step 10:

[1612] Users collect real-world factory information via their terminals and send it to the server.

[1613] Input: Real-world factory information (ongoing construction projects, infrastructure status, etc.)

[1614] Output: Consolidated factory data

[1615] How it works: Users input data about ongoing construction projects and existing infrastructure status into their terminals and send it to the server, where it is integrated into the virtual factory database.

[1616] Step 11:

[1617] The server updates the actual factory layout and earthquake response measures based on the integrated data.

[1618] Input: Integrated factory data

[1619] Output: Updated factory layout and earthquake response plan

[1620] Specific operation: The server analyzes the integrated data and updates the latest factory layout and earthquake response measures. The updated contents are notified to the user via the terminal, and corrections and suggestions are made as necessary.

[1621] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1622] This invention relates to a system that uses landscape images to build a virtual city and provide disaster response and reconstruction support for earthquakes and other disasters, and also combines it with an emotion engine that recognizes the user's emotions. This system analyzes the user's emotions and optimizes the design of the virtual city and reconstruction plans based on those analyses.

[1623] 1. Collecting landscape images and training the model

[1624] Collection of landscape images

[1625] User: Uploads images of urban and natural landscapes that he has taken to the system.

[1626] Terminal: The user's terminal sends the uploaded landscape image to the server.

[1627] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[1628] Image recognition model training

[1629] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[1630] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[1631] 2. Building a Virtual City

[1632] Virtual city prototyping

[1633] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[1634] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[1635] Gathering and elaborating user requirements

[1636] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[1637] Terminal: User input is sent to the server in real time.

[1638] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[1639] Generate detailed 3D models

[1640] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[1641] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[1642] 3. Earthquake response simulation and reconstruction support

[1643] Earthquake scenario setting and simulation

[1644] User: Set up a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1645] Terminal: Sends the configured earthquake scenario to the server.

[1646] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[1647] Server: As a result of the simulation, it calculates the locations of evacuation routes and important support facilities and generates corresponding maps.

[1648] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1649] Creation of a recovery support plan

[1650] User: Requests the generation of a post-disaster recovery plan.

[1651] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[1652] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1653] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[1654] 4. Incorporating an Emotional Engine

[1655] emotion recognition

[1656] Users: Express emotions through facial expressions, voice, and text input while using the system.

[1657] Device: Captures the user's facial expression data with a camera, records voice data with a microphone, and collects text input data.

[1658] Server: Analyzes the data sent from the device and uses an emotion engine to recognize the user's emotions, such as relief, anxiety, satisfaction, etc.

[1659] Utilizing Emotional Data

[1660] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels a lot of anxiety, it provides a design and support plan that emphasizes safety.

[1661] On the device: Present the optimized design or plan to the user and collect feedback again.

[1662] 5. Integrating virtual cities with the real world

[1663] Real-World Data Collection

[1664] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[1665] Terminal: Sends collected information to the server.

[1666] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1667] Providing integrated data and updating urban planning

[1668] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1669] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[1670] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[1671] For example, a user uploads landscape images they have taken, and the system identifies them to generate a prototype of a virtual city. The user then sets up an earthquake scenario in the VR environment, runs a simulation, and checks evacuation routes and support points. During this process, the system analyzes the user's emotional data and provides an optimized earthquake response plan based on their sense of security or anxiety.

[1672] The processing flow will be explained below.

[1673] Step 1:

[1674] User: Uploads landscape images to the system.

[1675] Terminal: Sends the uploaded landscape image to the server.

[1676] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[1677] Step 2:

[1678] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[1679] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[1680] Step 3:

[1681] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[1682] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[1683] Step 4:

[1684] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[1685] Terminal: User input is sent to the server in real time.

[1686] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[1687] Step 5:

[1688] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[1689] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[1690] Step 6:

[1691] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1692] Terminal: Sends the configured earthquake scenario to the server.

[1693] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[1694] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[1695] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1696] Step 7:

[1697] User: Requests the generation of a post-disaster recovery plan.

[1698] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[1699] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1700] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[1701] Step 8:

[1702] Users: Express emotions naturally while using the system. Express emotions while interacting with and operating the system through facial expressions, voice, text input, etc.

[1703] Device: Captures the user's facial expressions with a camera, records their voice with a microphone, and collects the text they type. This data is sent to the server in real time.

[1704] Step 9:

[1705] Server: Analyzes the data sent from the device and uses an emotion engine to recognize the user's emotions, such as relief, anxiety, satisfaction, etc.

[1706] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. If the user feels a lot of anxiety, the server will adjust the design and reconstruction plan to emphasize safety.

[1707] Step 10:

[1708] Server: Presents the optimized design or plan to the user and again collects feedback.

[1709] Device: Displays updated designs and plans to the user in the VR environment and sends feedback to the server.

[1710] User: Review the optimized designs and plans provided and provide any necessary feedback.

[1711] Step 11:

[1712] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[1713] Terminal: Sends collected information to the server.

[1714] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1715] Step 12:

[1716] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1717] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[1718] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[1719] The above are the specific program processing steps in the system that combines the emotion engine. This invention makes it possible to design virtual cities and support reconstruction efforts that take into account the emotions of users.

[1720] Example 2

[1721] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1722] In modern society, disaster response and reconstruction support are important, especially in urban environments. However, conventional systems have had difficulty integrating real-world city data with virtual city data and providing optimal plans that take user emotions into account. There is also a need for an integrated platform that can simulate specific disaster scenarios in a virtual reality environment and generate optimal response measures while performing real-time emotion analysis.

[1723] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1724] In this invention, the server includes means for collecting, tagging, and classifying landscape images, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating the required resources, and presenting the reconstruction plan, means for analyzing user emotions and optimizing the virtual city design and reconstruction plan based on the recognized emotion data, means for collecting real city information and integrating it into a virtual city database, and means for updating the real city plan and earthquake response measures based on the integrated data. This enables the system to integrate real city data and virtual city data in real time and generate optimal disaster response and reconstruction plans that take user emotions into consideration.

[1725] A "landscape image" is a digital image of a city, nature, or other landscape.

[1726] "Tagging" is the process of assigning labels to images that indicate their characteristics, such as "mountain," "river," or "building."

[1727] "Classification" is the process of separating tagged images into different categories.

[1728] An "image recognition model" is a machine learning model that analyzes patterns in landscape images and automatically identifies their features.

[1729] "Virtual City Prototype" is the creation of an early stage city model in virtual space based on collected landscape images.

[1730] "Detailed 3D model" refers to a detailed three-dimensional city model that includes specific design information based on user requirements.

[1731] An "earthquake disaster scenario" is a hypothetical scenario that simulates earthquakes and their associated impacts that may occur under specific conditions.

[1732] "Earthquake simulation" is a simulation that applies an earthquake scenario to a virtual urban environment to analyze evacuation routes and support points in the event of an earthquake.

[1733] A "support point" is the location of a base where important support activities are carried out in the event of a disaster.

[1734] A "reconstruction support plan" is a detailed plan that includes the resources needed to rebuild a city after a disaster and a plan to rebuild infrastructure.

[1735] An "emotion engine" is software that analyzes emotions from a user's facial expressions, voice, and text input.

[1736] A "virtual reality environment" is a digital environment that allows users to experience an immersive virtual space.

[1737] "City information" refers to data about real cities that shows the state of buildings under construction and existing infrastructure.

[1738] A "database" is a system for systematically storing and managing landscape images, virtual city models, emotional data, etc.

[1739] "Integration" refers to combining multiple different data sets in order to manage and use them in a centralized manner.

[1740] This invention relates to a system that uses landscape images to build a virtual city and provide disaster response and reconstruction support, as well as a system that combines an emotion engine that recognizes the user's emotions. The system analyzes the user's emotions and optimizes the design of the virtual city and reconstruction plans based on those analyses.

[1741] 1. Collecting landscape images and training the model

[1742] Collection of landscape images

[1743] Users: Users upload images of urban and natural landscapes taken with their smartphones or digital cameras, thereby collecting data on realistic urban environments.

[1744] Terminal: The user's terminal sends these landscape images to the server.

[1745] Server: The server stores the received landscape images in a database and tags and classifies them. Specific tags include "mountain," "river," and "building." This process uses image recognition services such as Google Cloud Vision API.

[1746] Image recognition model training

[1747] Server: The server uses landscape images stored in a database to train an image recognition model. Specific software used includes TensorFlow and Keras.

[1748] Server: As a result of the training, a model that can accurately identify landscape images is generated. This model is saved in a database and used in subsequent processing.

[1749] 2. Building a Virtual City

[1750] Virtual city prototyping

[1751] Server: Utilizing a trained image recognition model, the server generates a prototype of a virtual city by analyzing landscape image data stored in a database. It uses 3D modeling software such as Unity.

[1752] Server: Apply earthquake-resistant design standards, such as seismic strength and evacuation routes, to the prototype and make adjustments.

[1753] Gathering and elaborating user requirements

[1754] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, setting of residential areas) through a web form.

[1755] Terminal: Sends user-entered data to the server in real time.

[1756] Server: Analyzes the received requirements and incorporates them into the virtual city prototype.

[1757] Generate detailed 3D models

[1758] Server: Generate a detailed 3D model of the virtual city using 3D modeling tools such as Blender or Maya.

[1759] Server: The generated 3D models are stored in a database and exported to a VR environment (e.g., Oculus Rift compatible).

[1760] 3. Earthquake response simulation and reconstruction support

[1761] Earthquake scenario setting and simulation

[1762] User: Set up a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1763] Device: Sends settings to the server via the VR headset.

[1764] Server: Using simulation tools such as Unity, we run earthquake simulations in a virtual city environment, and analyze the results to determine evacuation routes and the locations of important support points.

[1765] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[1766] Creation of a recovery support plan

[1767] User: Requests a post-disaster recovery plan from the system.

[1768] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.). This utilizes Unity's analysis functions.

[1769] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[1770] Device: After viewing the reconstruction support plan in the VR environment, the user sends specific feedback to the server.

[1771] 4. Incorporating an Emotional Engine

[1772] emotion recognition

[1773] User: Expresses emotions through facial expressions, voice, and text input while using the system.

[1774] Device: Captures facial expressions with a camera, records audio with a microphone, and collects text input data.

[1775] Server: Analyzes the collected data using an emotion recognition API (e.g., Microsoft Azure Emotion API) to determine the user's emotions. For example, it identifies "relief," "anxiety," "satisfaction," etc.

[1776] Utilizing Emotional Data

[1777] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels anxious, it provides a design with enhanced safety and a support plan.

[1778] On the device: Present the optimized design or plan to the user and collect feedback again.

[1779] 5. Integrating virtual cities with the real world

[1780] Real-World Data Collection

[1781] Users: Enter real-world city information through the application, including the status of buildings under construction and the state of existing infrastructure.

[1782] Terminal: Sends collected information to the server.

[1783] Server: Integrates real-world data into the virtual city database and centrally manages data on the virtual city and real city.

[1784] Providing integrated data and updating urban planning

[1785] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1786] Device: Provides updated information to users, notifying them of the latest urban planning information and earthquake response measures.

[1787] User: Checks the provided information, sends necessary feedback to the server, and considers specific measures and actions based on this information.

[1788] Examples of specific examples and prompts

[1789] For example, a user uploads landscape images they have taken, and the system identifies them to generate a prototype of a virtual city. The user then sets up an earthquake scenario in the VR environment and runs a simulation to check evacuation routes and support points. During this process, the system analyzes the user's emotional data and provides an optimized earthquake response plan based on their sense of security or anxiety.

[1790] Example prompt sentence:

[1791] Please upload this landscape image.

[1792] "Please enter the requirements for your virtual city."

[1793] "Set up an earthquake scenario."

[1794] "Use your camera and microphone to express your emotions."

[1795] "Please enter real city information."

[1796] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1797] Step 1: Collecting landscape images

[1798] Users: Users upload images of city and natural scenery taken with their smartphones or digital cameras.

[1799] Input: Landscape image file (e.g. JPEG, PNG format)

[1800] Terminal: The user's terminal sends the uploaded landscape image to the server.

[1801] Server: The server stores the received landscape images in a database. It also uses a Python script to automatically tag and classify the images as "mountains," "rivers," "buildings," etc. This process uses the Google Cloud Vision API.

[1802] Output: A tagged and classified landscape image database

[1803] How it works: Users provide cityscape images following the prompt "Please upload," and the server stores and automatically tags them.

[1804] Step 2: Training the image recognition model

[1805] Server: The server uses the landscape images stored in the database to train an image recognition model using TensorFlow and Keras.

[1806] Input: A tagged and classified landscape image database

[1807] Data processing: Extract features from image data and train a model using machine learning algorithms.

[1808] Output: A trained, highly accurate image recognition model

[1809] How it works: The server retrieves landscape images from the database and trains them with the TensorFlow library to generate an image recognition model.

[1810] Step 3: Prototype the virtual city

[1811] Server: The server uses a trained image recognition model to analyze landscape image data from the database and generate a prototype of the virtual city. It uses 3D modeling software such as Unity.

[1812] Input: trained image recognition model, landscape image database

[1813] Data processing: Based on landscape images, prototypes are generated using 3D modeling software and seismic design criteria are applied.

[1814] Output: A prototype of a seismically designed virtual city

[1815] How it works: The server recognizes buildings and natural elements from landscape images and builds an initial virtual city model using Unity or Blender.

[1816] Step 4: Gathering and elaborating user requirements

[1817] User: The user fills out a web form with the requirements for the virtual city, including the placement of public facilities, the design of the transport infrastructure, and the setting of residential areas.

[1818] Input: User-entered requirement data (web form)

[1819] Terminal: The user terminal sends inputs to the server in real time.

[1820] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[1821] Output: Prototype reflecting user requirements

[1822] Specific operation: The user enters data into a web form through a prompt that says, "Please enter the requirements for your virtual city." The server analyzes this data and reflects it in the prototype.

[1823] Step 5: Generate a detailed 3D model

[1824] Server: The server uses 3D modeling tools such as Blender or Maya to generate a detailed 3D model of the virtual city based on the user requirements.

[1825] Input: Prototype reflecting user requirements

[1826] Data manipulation: Use 3D tools to refine and perfect your prototype.

[1827] Output: A detailed 3D model of a virtual city

[1828] What it does: The server takes requirements from the user and generates a detailed 3D model in Blender or Maya.

[1829] Step 6: Setting up and simulating earthquake scenarios

[1830] User: Set a specific earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[1831] Input: Earthquake scenario (VR environment)

[1832] Device: Sends settings to the server via the VR headset.

[1833] Server: Using simulation tools such as Unity, we run earthquake simulations in a virtual city environment and analyze the results.

[1834] Data processing: Analyze the simulation results and calculate the locations of evacuation routes and support points.

[1835] Output: Simulation results including evacuation routes and assistance points

[1836] Specific operation: The user puts on the VR headset and sets up the scenario according to the prompt, "Please set up the earthquake scenario." The server receives the data and runs the simulation.

[1837] Step 7: Generate a recovery plan

[1838] User: The user requests the system to generate a post-disaster recovery plan.

[1839] Input: Request (instructions to the system)

[1840] Server: Analyzes the impact of the earthquake based on the simulation results and calculates the required resources. Utilizing Unity's analysis functions.

[1841] Data processing: Calculate the necessary resources (building materials, personnel, equipment, etc.) and generate an efficient reconstruction support plan.

[1842] Output: Recovery assistance plan, infrastructure reconstruction plan, housing reconstruction plan

[1843] Specific operation: The user requests, "Please generate a post-disaster reconstruction support plan," and the server calculates the necessary resources and generates the support plan based on this.

[1844] Step 8: Emotion Recognition

[1845] User: The user expresses their emotions through facial expressions, voice, and text input while using the system.

[1846] Input: facial expressions, voice, text data

[1847] Device: Uses a camera and microphone to capture facial expressions and voice, and collects text input data.

[1848] Server: Analyze user emotions from collected data using an emotion recognition API (e.g., Microsoft Azure Emotion API).

[1849] Data processing: Facial expressions, voice, and text data are analyzed using an emotion engine to identify emotions (e.g., relief, anxiety).

[1850] Output: Recognized emotion data

[1851] Specific behavior: While using the system, the user follows the prompt: "Please use your camera and microphone to express your emotions." The server analyzes the captured data and identifies the emotions.

[1852] Step 9: Leverage sentiment data

[1853] Server: Optimizes the design and reconstruction plan of the virtual city based on the recognized emotion data. For example, if the user feels anxious, it provides a design and support plan that enhances safety.

[1854] Input: Recognized emotion data, virtual city data

[1855] Data processing: Adjust and optimize designs and plans based on sentiment data.

[1856] Output: Optimized virtual city design and reconstruction support plan

[1857] Specific operation: The server takes in the emotional data, makes design changes, for example to increase the sense of security, and shows them to the user.

[1858] Step 10: Real-world data collection and integration

[1859] User: Uses the application to input real-world city information (status of buildings under construction, existing infrastructure, etc.).

[1860] Input: Real city information

[1861] Terminal: Sends collected information to the server.

[1862] Server: Integrates real-world data into the virtual city database and centrally manages real and virtual data.

[1863] Output: Integrated city database

[1864] Specific operation: The user inputs real city information according to the application's prompts, and the server receives it and integrates it with the virtual city data.

[1865] Step 11: Providing integrated data and updating urban plans

[1866] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1867] Input: Integrated City Database

[1868] Data processing: Analyze the integrated data and update real-world urban planning and disaster response measures.

[1869] Output: Updated urban planning information, earthquake response measures

[1870] Specific operation: The server analyzes the integrated data, generates the latest urban planning information and earthquake response measures, and notifies the user.

[1871] The specific actions at each step ensure that the entire system functions efficiently, enabling users to obtain optimal disaster response and recovery plans in real time.

[1872] (Application example 2)

[1873] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1874] Although there are systems that build virtual cities based on real-world landscape images and provide disaster response and reconstruction support, conventional systems are not optimized to take user emotions into account, making it difficult to provide designs and plans that provide the most peace of mind and satisfaction to users. Furthermore, virtual store designs are not optimized based on user emotions, making it difficult to provide the optimal shopping experience tailored to each individual user.

[1875] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting landscape images and tagging and classifying them, means for training an image recognition model using the landscape images, means for generating a virtual city prototype using the trained image recognition model, means for accepting user requirements and reflecting them in the virtual city prototype to generate a detailed 3D model, means for setting an earthquake scenario and performing an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for generating a post-earthquake reconstruction support plan, calculating required resources, and presenting the reconstruction plan, means for collecting real city information and integrating it into a virtual city database, means for updating real city plans and earthquake response measures based on the integrated data, means for analyzing user emotion data using an emotion recognition engine and optimizing the virtual city design and reconstruction plan based on the emotion data, and means for generating a virtual store based on landscape images uploaded by a user, adjusting the design and product placement of the virtual store based on the emotion data, and providing the user with the virtual store optimized in the virtual reality environment. This will enable disaster response and reconstruction support plans to be provided that take into account the user's emotions, and will also enable the creation of an optimal shopping experience in virtual stores.

[1876] "Landscape images" are photographs or video footage of natural or urban landscapes.

[1877] "Tagging" is the act of assigning keywords and classification information to data or images.

[1878] "Classification" is the act of separating data or information into specific categories or groups.

[1879] An "image recognition model" is a mathematical model that uses machine learning algorithms to analyze images and identify their content.

[1880] A "virtual city" is an urban environment generated by computer simulation.

[1881] A "prototype" is an early model or prototype of a new system or product.

[1882] "User requirements" refers to the functions and conditions that users desire for the system.

[1883] A "detailed 3D model" is a model of an object or environment that is recreated in detail in three-dimensional space.

[1884] An "earthquake disaster scenario" is a simulation scenario that assumes the occurrence of an earthquake or related disaster.

[1885] "Earthquake simulation" is a system that uses a computer to recreate earthquakes and related disasters and analyze their impact.

[1886] An "evacuation route" is a route that people can use to safely evacuate in the event of a disaster.

[1887] "Support points" are important locations where support activities are carried out during disasters.

[1888] A "reconstruction support plan" is a plan to support the reconstruction of a region after an earthquake or other disaster.

[1889] "Resources" are the resources and materials required to run a system or project.

[1890] "Real city information" refers to data and contextual information about real cities.

[1891] The "Virtual City Database" is a database system that manages and stores various data related to virtual cities.

[1892] An "emotion recognition engine" is a system that analyzes a user's facial expressions, voice, text input, etc. to identify their emotions.

[1893] A "virtual store" is a virtual store built on the Internet or in a virtual reality space.

[1894] "Design" is the act of designing objects or environments with shapes and structures that have specific purposes and functions.

[1895] "Product placement" refers to the act of deciding how to place products in a store.

[1896] A "virtual reality environment" is a three-dimensional space generated using computer technology in which a user can act or experience something.

[1897] This system uses landscape images to build a virtual city, recognizes the user's emotions, and provides optimized designs and plans based on those emotions. This system is targeted at disaster response and reconstruction support, as well as virtual store generation and optimization.

[1898] System configuration

[1899] This system uses the following hardware and software:

[1900] Hardware

[1901] Smartphone (camera and microphone for capturing landscape images and emotion recognition)

[1902] Smart glasses (optional, for displaying virtual reality environments and emotion recognition)

[1903] Server (data processing and storage)

[1904] software

[1905] Image recognition model (convolutional neural network)

[1906] Emotion recognition engine (machine learning model)

[1907] Virtual Reality (VR) Engine

[1908] System Operation

[1909] 1. Collecting and tagging landscape images

[1910] Users use their smartphones to take pictures of urban or natural landscapes and upload them to the system. The server receives the uploaded images and tags and classifies them, adding tags such as "mountain," "river," and "building."

[1911] 2. Training the image recognition model

[1912] The server trains an image recognition model using landscape images stored in a database, using a convolutional neural network (CNN) to generate a highly accurate model for identifying landscape images.

[1913] 3. Prototype generation of virtual city

[1914] Using a trained image recognition model, the system analyzes landscape image data to generate a prototype of a virtual city, taking into account earthquake resistance standards and evacuation routes.

[1915] 4. Reflecting user requirements and generating detailed 3D models

[1916] Users input the requirements for a virtual city into the system, such as the layout of public facilities, the design of transportation infrastructure, the setting of residential areas, etc. The server analyzes the received requirements, incorporates them into the virtual city prototype, and generates a detailed 3D model.

[1917] 5. Earthquake Disaster Simulation and Generation of Recovery Support Plans

[1918] Based on the earthquake scenario set by the user, the server runs an earthquake simulation in a virtual city environment. Based on the simulation results, it calculates evacuation routes and support points, and generates a reconstruction support plan. It also calculates the necessary resources (building materials, personnel, equipment, etc.).

[1919] 6. Emotion Recognition and Optimization

[1920] When using the system, the camera and microphone on the smartphone or smart glasses are used to collect the user's facial expressions and speech. An emotion recognition engine analyzes this data and identifies the user's feelings of relief, anxiety, satisfaction, etc. The server then uses the recognized emotion data to optimize the design of the virtual city and reconstruction plans.

[1921] 7. Virtual store generation and optimization

[1922] The server generates a virtual store based on landscape images uploaded by users. The design and product layout of the virtual store are adjusted based on emotional data from users while using the system. The optimized virtual store is provided to users in a virtual reality environment, where they can experience and shop.

[1923] Prompt Sentence Examples

[1924] "Log in to this virtual store app and upload a landscape image taken with your smartphone. A virtual store will be generated based on that image. Next, walk around the store freely and look at the products. Actively express emotions such as joy, excitement, or anxiety when you pick up a particular product. The emotion recognition engine analyzes these emotions and suggests optimal store design and product placement."

[1925] The above is an embodiment of the present invention. This system makes it possible to provide disaster response and reconstruction support plans that take into account the user's emotions, and also to realize an optimal shopping experience in a virtual store.

[1926] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1927] Step 1:

[1928] Collecting and tagging landscape images

[1929] Users use their smartphones to take images of urban or natural landscapes and upload them to the system. The server receives the uploaded images and tags and classifies them. The input is the landscape image taken by the user, and the output is the tagged and classified landscape image. Specifically, the server uses a convolutional neural network (CNN) to extract image features and assign tags such as "mountain," "river," and "building."

[1930] Step 2:

[1931] Image recognition model training

[1932] The server trains an image recognition model using landscape images stored in a database. The input is tagged and classified landscape images, and the output is a trained image recognition model. Specifically, the server trains a CNN using a large landscape image dataset, which generates a highly accurate model for accurately identifying landscape images.

[1933] Step 3:

[1934] Virtual city prototyping

[1935] The server uses a trained image recognition model to analyze input landscape image data and generate a prototype of the virtual city. The output is an initial prototype of the virtual city. Specifically, the server extracts features from the landscape image data, generates 3D models of buildings and terrain, and uses these to create the basic structure of the virtual city.

[1936] Step 4:

[1937] Reflecting user requirements and generating detailed 3D models

[1938] Users input their requirements for a virtual city into the system, including the layout of public facilities, the design of transportation infrastructure, and the establishment of residential areas. The server receives this information, analyzes it, and applies it to a prototype of the virtual city, generating a detailed 3D model. The input is the user's requirements, and the output is a 3D model based on the user's requirements. Specifically, the server adjusts the layout of the prototype based on the conditions entered by the user and runs an algorithm to generate a detailed model.

[1939] Step 5:

[1940] Earthquake disaster simulation and generation of reconstruction support plans

[1941] The server runs an earthquake simulation in a virtual city environment, using an earthquake scenario set by the user as input. The input is the earthquake scenario, and the output is evacuation routes and support points based on the simulation results. The server analyzes the simulation results, calculates the resources required to generate a post-earthquake reconstruction support plan, and presents the reconstruction plan. Specifically, it simulates the impact within the virtual city based on parameters such as the location, time, and seismic intensity of the earthquake, and identifies evacuation routes and key points.

[1942] Step 6:

[1943] Emotion Recognition and Optimization

[1944] When a user uses the system, the system collects their facial expressions and speech using a camera and microphone installed on their smartphone or smart glasses. The input is the user's facial expression and voice data, and the output is analyzed emotional data. The server analyzes this data using an emotion recognition engine to identify the user's feelings of relief, anxiety, satisfaction, etc. Furthermore, it optimizes the design and reconstruction plan of the virtual city based on the emotional data. Specifically, it runs an algorithm to apply a design and layout that makes the user feel at ease.

[1945] Step 7:

[1946] Virtual store generation and optimization

[1947] The server generates a virtual store based on landscape images uploaded by users. The input is landscape images taken by users and emotional data, and the output is an optimized virtual store design. The server adjusts the virtual store design and product layout based on the emotional data. The optimized virtual store is provided to users in a virtual reality environment, where they can experience and shop. Specifically, the system analyzes emotional data in real time as users browse products, and sequentially optimizes product layout and design.

[1948] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1949] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1950] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1951] [Fourth embodiment]

[1952] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1953] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1954] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1955] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1956] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1957] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1958] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1959] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1960] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1961] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1962] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1963] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1964] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1965] The following system configuration and operation will be described as an embodiment of the invention: This system uses landscape images to build a virtual city and to respond to earthquake disasters and provide reconstruction support.

[1966] 1. Collecting landscape images and training the model

[1967] Collection of landscape images

[1968] User: Uploads images of urban and natural landscapes that he has taken to the system.

[1969] Terminal: The user's terminal sends the uploaded landscape image to the server.

[1970] Server: The received landscape images are stored in a database, and are tagged and classified. For example, they are organized by adding tags such as "mountain," "river," and "building."

[1971] Image recognition model training

[1972] Server: Uses the stored landscape images to train an image recognition model such as a convolutional neural network (CNN).

[1973] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[1974] 2. Building a Virtual City

[1975] Virtual city prototyping

[1976] Server: Uses trained image recognition models to generate terrain and infrastructure prototypes from landscape image data.

[1977] Server: This prototype will be designed to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[1978] Gathering and elaborating user requirements

[1979] User: Enters the requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting of residential areas) into the system.

[1980] Terminal: User input is sent to the server in real time.

[1981] Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements and exports it to the VR environment.

[1982] 3. Earthquake response simulation and reconstruction support

[1983] Setting up a disaster scenario

[1984] User: Set up a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[1985] Terminal: Sends scenario setting information to the server.

[1986] Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[1987] Creation of a recovery support plan

[1988] User: Submits a request for a post-disaster recovery plan.

[1989] Server: Analyzes the impact of the earthquake and calculates the necessary resources (e.g., building materials, personnel, and equipment). Generates an optimal reconstruction support plan and presents it to the user.

[1990] Device: The user checks the reconstruction assistance plan in a VR environment.

[1991] 4. Integrating virtual cities with the real world

[1992] Real-World Data Collection

[1993] Users: Enter real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[1994] Terminal: Sends collected information to the server.

[1995] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[1996] Providing integrated data and updating urban planning

[1997] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[1998] Device: Provide users with up-to-date planning information and gather feedback.

[1999] User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[2000] For example, a user can upload landscape images they have taken, and the system will analyze them and generate a prototype of a virtual city. After that, the user can set up an earthquake scenario in the VR environment and check the generated simulation results, which will enable efficient earthquake response and reconstruction support.

[2001] The processing flow will be explained below.

[2002] Step 1:

[2003] User: Uploads landscape images to the system.

[2004] Terminal: Sends the uploaded landscape image to the server.

[2005] Server: Stores the received landscape images in a database and performs tagging and classification. For example, it organizes them by adding tags such as "mountain," "river," and "building."

[2006] Step 2:

[2007] Server: Trains an image recognition model (e.g., a convolutional neural network) using landscape images stored in a database.

[2008] Server: As a result of the learning process, a reliable model for identifying landscape images is generated and stored in a database.

[2009] Step 3:

[2010] Server: Using a trained image recognition model, analyzes landscape image data and generates a prototype of a virtual city.

[2011] Server: Adjust the prototype by taking into account earthquake-resistant design standards (seismic resistance, evacuation routes, etc.).

[2012] Step 4:

[2013] User: Enters the requirements for the virtual city into the system, including specifying the placement of public facilities, the design of the transportation infrastructure, and the setting of residential areas.

[2014] Terminal: Sends the requirements entered by the user to the server in real time.

[2015] Server: Analyzes the received requirements and incorporates them into a prototype of the virtual city.

[2016] Step 5:

[2017] Server: Generates a detailed 3D model of a virtual city based on user requirements.

[2018] Server: Stores the generated 3D models in a database and exports them to the VR environment.

[2019] Step 6:

[2020] User: Set up an earthquake scenario (e.g., location of epicenter, seismic intensity, time of occurrence) in the VR environment.

[2021] Terminal: Sends the configured earthquake scenario to the server.

[2022] Server: Runs earthquake simulations in a virtual city environment and analyzes the results.

[2023] Server: As a result of the simulation, it calculates evacuation routes and the locations of important support facilities, and generates a corresponding map.

[2024] Terminal: Displays the corresponding map generated in the VR environment and the simulation results to the user.

[2025] Step 7:

[2026] User: Requests the generation of a post-disaster recovery plan.

[2027] Server: Analyzes the impact of the earthquake and calculates the resources required (building materials, personnel, equipment, etc.).

[2028] Server: Generates efficient reconstruction assistance plans and presents necessary infrastructure and housing reconstruction plans.

[2029] Device: The user checks the reconstruction support plan in a VR environment and sends specific feedback to the server.

[2030] Step 8:

[2031] User: Enters real-world city information (e.g., the status of buildings under construction, the state of existing infrastructure) through the application.

[2032] Terminal: Sends collected information to the server.

[2033] Server: Integrates real-world data into the virtual city database and centrally manages data on virtual and real cities.

[2034] Step 9:

[2035] Server: Updates real-world urban planning and earthquake response measures based on the integrated data.

[2036] Device: Provides updated information to users and notifies them of the latest planning information and disaster response measures.

[2037] User: Checks the provided information and provides necessary feedback. Users also consider specific measures and actions based on this information.

[2038] The above are the processing steps of the specific program for carrying out the invention.

[2039] Example 1

[2040] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2041] Conventional urban planning and earthquake response systems have difficulty integrating and operating real-world data with virtual environments, resulting in a lack of integrated means for efficient urban planning, disaster simulation, and reconstruction support. Furthermore, it is difficult to update user-provided information in real time in the virtual city, making it difficult to effectively use these systems in situations where a rapid and accurate response is required.

[2042] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2043] In this invention, the server includes means for collecting landscape images taken by users and tagging and classifying them, means for training an image recognition model such as a convolutional neural network using the landscape image data, means for generating a virtual city prototype using the trained image recognition model, means for inputting user requirements, means for converting the virtual city prototype into a detailed 3D model reflecting the user requirements, means for setting an earthquake scenario and running an earthquake simulation in a virtual city environment to provide evacuation routes and support points, means for using the server to generate a post-earthquake reconstruction support plan, calculate the required resources, and present the reconstruction plan, means for collecting real-world city information and integrating it into a virtual city database, and means for updating real-world city plans and earthquake response measures based on the integrated data. This enables information provided by users to be quickly and accurately reflected in the virtual city, enabling efficient urban planning, earthquake response, and reconstruction support.

[2044] "User" refers to a person who uploads landscape images to the system, inputs requirements, and checks and uses the simulation results and reconstruction plans.

[2045] "Terminal" refers to a device that a user connects to and operates the system, and is used to upload landscape images, input requirements, and check simulation results.

[2046] The "server" refers to the central processing unit that is the back-end computing resource that supports the entire system and performs tasks such as collecting images, tagging, classifying, learning, generating prototypes, running simulations, generating reconstruction assistance plans, and integrating data.

[2047] "Landscape images" refer to image data of urban and natural landscapes that users take and upload to the system.

[2048] "Tagging" refers to the process of categorizing collected landscape images by assigning keywords such as "mountain," "river," and "building."

[2049] "Classification" refers to the process of organizing tagged landscape images into categories and storing them in a database.

[2050] An "image recognition model" refers to a machine learning model trained to identify and classify landscape images using a convolutional neural network (CNN) or similar.

[2051] "Virtual city" refers to a virtual urban environment created based on landscape image data, generated using a trained image recognition model.

[2052] "Prototype" refers to an early model or structure of a virtual city, a temporary mockup before being refined with further user requirements and data.

[2053] "User requirements" refers to the various specific requests and specifications for the virtual city that the user inputs into the system.

[2054] "Detailed 3D model" refers to a detailed three-dimensional virtual city model that is generated as a result of reflecting user requirements.

[2055] "Earthquake scenario" refers to the conditions for an earthquake to occur (e.g., epicenter, seismic intensity, time period) set in a virtual city environment.

[2056] "Earthquake simulation" refers to the process of simulating the impact and evacuation routes of an earthquake occurring within a virtual city environment based on a set earthquake scenario.

[2057] An "evacuation route" refers to a route within a virtual city that is set up for safe evacuation in an earthquake simulation.

[2058] "Support points" refer to points or locations where evacuees can receive support in earthquake simulations.

[2059] "Reconstruction Support Plan" refers to a plan that includes the resources (e.g., building materials, personnel, and equipment) needed to rebuild a virtual city after a disaster.

[2060] "Real-world urban information" refers to data such as the current state of cities and ongoing construction projects in the real world.

[2061] A "virtual city database" refers to a database for centrally managing real city information and virtual city information.

[2062] "Urban planning" refers to plans and policies for the future development and maintenance of virtual and real cities.

[2063] "Earthquake response measures" refers to plans that include actions, procedures, and preparations to be taken in the event of an earthquake.

[2064] This invention is a system for constructing a virtual city using landscape images and for responding to earthquake disasters and supporting reconstruction efforts. How each step is carried out will be described in detail below.

[2065] Collecting landscape images and learning models

[2066] Collection of landscape images

[2067] 1. User: Takes images of urban or natural scenery with a smartphone or digital camera and uploads them to the system using a dedicated application.

[2068] 2. Device: The user's device (e.g., smartphone or PC) sends the uploaded landscape images to the server.

[2069] 3. Server: The received landscape images are stored in a database and tagged and classified. Tags are automatically assigned using an image recognition algorithm, such as "mountain," "river," and "building." Classification is based on these tags.

[2070] Image recognition model training

[2071] 1. Server: Using the stored landscape images, an image recognition model such as a convolutional neural network (CNN) is trained using a machine learning library such as TensorFlow or PyTorch.

[2072] 2. Server: As a result of the training, a reliable model for identifying landscape images is generated and stored in a database.

[2073] Building a Virtual City

[2074] Virtual city prototyping

[2075] 1. Server: Using trained image recognition models, prototypes of terrain and infrastructure are generated from landscape image data. For example, prototypes are created using game engines such as Unity or Unreal Engine.

[2076] 2. Server: This prototype is applied with algorithms to reflect earthquake-resistant design standards (seismic strength, emergency evacuation routes, etc.).

[2077] Gathering and elaborating user requirements

[2078] 1. User: Uses a dedicated application or web interface to input specific requirements for the virtual city (e.g., placement of public facilities, design of transportation infrastructure, and setting up residential areas) into the system.

[2079] 2. Terminal: Sends user input to the server in real time.

[2080] 3. Server: Analyzes user requirements, incorporates them into a prototype, and generates a detailed 3D model of the virtual city based on the requirements, which is then exported to the VR environment.

[2081] Earthquake response simulation and reconstruction support

[2082] Setting up a disaster scenario

[2083] 1. User: Set a specific earthquake scenario (e.g., epicenter, seismic intensity, time period) in the VR environment.

[2084] 2. Terminal: Sends the configured scenario information to the server.

[2085] 3. Server: Runs the simulation in a virtual city environment and provides evacuation routes and support points.

[2086] Creation of a recovery support plan

[2087] 1. User: Submits a request for a post-disaster recovery plan.

[2088] 2. Server: The system analyzes the impact of the earthquake and calculates the required resources (e.g., building materials, personnel, equipment) using optimization models and simulation tools.

[2089] 3. Server: Generates the optimal reconstruction assistance plan and presents it to the user.

[2090] 4. Device: The user checks the reconstruction assistance plan in a VR environment.

[2091] Integrating virtual cities with the real world

[2092] Real-World Data Collection

[2093] 1. User: Enters real-world city information (e.g., ongoing construction projects, existing infrastructure status) through the application.

[2094] 2. Terminal: Sends the collected information to the server.

[2095] 3. Server: Integrates real-world data into the virtual city database and centrally manages virtual city and real-world data.

[2096] Providing integrated data and updating urban planning

[2097] 1. Server: Updates real-world urban planning and disaster response measures based on the integrated data.

[2098] 2. Device: Provide users with up-to-date planning information and gather feedback.

[2099] 3. User: Send feedback based on the information provided and make any necessary corrections or suggestions.

[2100] Specific examples

[2101] Users upload landscape images taken with their smartphones to the system through a dedicated application. The server classifies the images and generates a prototype of the virtual city using Unity. The user then inputs further detailed requirements, and the server generates a detailed 3D model based on those requirements. The user then sets up an earthquake scenario in the VR environment, and the server runs the earthquake simulation and provides evacuation routes and support points.

[2102] Prompt Sentence Examples

[2103] "Please explain in detail the process by which a user uploads landscape images they have taken, and the system classifies them to generate a prototype of a virtual city. Also, please explain with concrete examples the steps by which a user sets up an earthquake scenario in a VR environment and checks the results."

[2104] The flow of the identification process in the first embodiment will be described with reference to FIG. ...

Claims

1. A means for collecting, tagging, and classifying landscape images; means for training an image recognition model using the landscape image; A means for generating a prototype of a virtual city using the trained image recognition model; a means for receiving user requirements and incorporating them into a prototype of the virtual city to generate a detailed 3D model; A means for setting up earthquake disaster scenarios and running earthquake disaster simulations in a virtual city environment to provide evacuation routes and support points; A means for generating a post-earthquake reconstruction assistance plan, calculating the required resources, and presenting the reconstruction plan; A means of collecting real city information and integrating it into a virtual city database; A means for updating actual city plans and earthquake disaster countermeasures based on the integrated data; A system including:

2. 10. The system of claim 1, wherein the means for setting up an earthquake scenario uses a virtual reality environment.

3. 2. The system according to claim 1, wherein the means for collecting real city information is performed through an application.

Images