system

Abstract

We provide the system. [Solution] A means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data, Means for preprocessing the collected data, A means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using preprocessed data, A means of receiving detailed information about sparsely populated buildings from users, A means of inputting received detailed information into a generating AI model to generate renovation ideas, A means for generating a completed design image using an image generation model based on the generated idea, A means of returning the generated renovation ideas and design images to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the spread of telecommuting during the COVID-19 pandemic, the extreme concentration in urban areas is being alleviated, but the depopulation of rural areas remains a major issue. When the working population in rural areas decreases, it becomes difficult to maintain and manage supermarkets, hospitals, city halls, etc. that support local life, increasing the likelihood of these facilities being closed. Conventional approaches lack means to find appropriate renovation ideas for depopulated buildings and have not established a specific methodology for rediscovering and utilizing the charm of the region. To solve such problems, there is a demand for a system that supports the renovation of buildings in depopulated areas by utilizing generative AI and realizes regional revitalization.

Means for Solving the Problems

[0005] This invention provides a system for collecting and preprocessing regional population data, traffic data, nearby facility data, and pedestrian flow data. The system uses the preprocessed data to train a generative AI model that generates optimal utilization ideas for sparsely populated buildings, and also includes means for receiving detailed information about sparsely populated buildings from users. The received detailed information is input into the generative AI model to generate renovation ideas, and based on these ideas, an image generation model generates a design image of the completed building. This design image can also be used as promotional material, and means for returning it to the user are also included. Furthermore, the system provides a user interface and means for users to input detailed information about sparsely populated buildings, comprehensively supporting regional revitalization.

[0006] "Regional population data" refers to statistical information about the population within a specific region, such as the total number of people, age groups, and gender.

[0007] "Traffic data" refers to data about traffic conditions within a specific area, including information such as congestion information, traffic volume, and public transport usage.

[0008] "Neighborhood facility data" refers to data showing the location and usage status of essential facilities such as supermarkets, schools, hospitals, and parks located within a specific area.

[0009] "People flow data" refers to data that shows the movement patterns and length of stay of people within a specific area.

[0010] "Preprocessing" refers to the processes performed after data collection to prepare the data for analysis and learning, such as imputing missing values, removing outliers, and standardizing the data.

[0011] A "generative AI model" is artificial intelligence trained to perform a specific task (in this case, generating renovation ideas for sparsely populated buildings) using collected data.

[0012] A "renovation idea" refers to a concrete proposal that presents new ways to utilize or remodel sparsely populated buildings.

[0013] An "image generation model" is a model that generates new images based on given information, and is a tool for creating designs and visuals using AI technology.

[0014] A "user interface" refers to a screen or input form that allows users to enter detailed information about a sparsely populated building and request the generation of renovation ideas. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[0037] server

[0038] Data Acquisition and Preprocessing

[0039] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. It then performs preprocessing on the collected data, such as imputing missing values, removing outliers, and standardizing the data. This prepares the data for training.

[0040] Training of Generative AI Models

[0041] The server trains a generative AI model using preprocessed data. This AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used for training, and the data is divided into training and validation data for model training. Model evaluation is also performed at this stage to verify its performance.

[0042] Processing the request

[0043] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. This information includes the building's location and current usage. The server inputs this information into an AI model to generate ideas for its utilization.

[0044] Idea generation and promotional material creation

[0045] Based on the generated utilization ideas, the server uses an image generation model to create images and videos that visually represent the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated utilization ideas and promotional materials are then returned to the terminal.

[0046] terminal

[0047] User Interface

[0048] The terminal provides users with an interface for entering detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, and their wishes and conditions regarding renovations.

[0049] Send Request

[0050] To send the information entered by the user to the server, the terminal formats the data and sends the request.

[0051] Displaying Results

[0052] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[0053] User

[0054] Entering information

[0055] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This includes, for example, the building's location, size, and current use. Users can also enter their wishes and conditions regarding renovations.

[0056] Idea verification

[0057] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0058] Specific example

[0059] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information of the vacant school building, its current condition (with photos), and their desired use (for example, to use it as a community center) through the terminal interface.

[0060] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated space. These results are returned to the terminal, allowing the user to review them and develop a concrete renovation plan.

[0061] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed regularly with the aim of obtaining accurate and up-to-date information.

[0065] Step 2:

[0066] The server performs preprocessing on the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. This prepares a dataset suitable for subsequent AI model training.

[0067] Step 3:

[0068] The server trains a generative AI model using preprocessed data. It splits the data into training data and validation data, trains the model using a deep learning framework (e.g., TensorFlow or PyTorch), and evaluates the model's performance.

[0069] Step 4:

[0070] Users input detailed information about sparsely populated buildings (location, current photos, renovation requests and conditions, etc.) through the terminal's user interface. Once the input is complete, they submit a request for renovation idea generation.

[0071] Step 5:

[0072] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[0073] Step 6:

[0074] The server receives detailed information about sparsely populated buildings sent from the terminal. The received information includes the building's location and details on how to use it.

[0075] Step 7:

[0076] The server inputs the received information into a generating AI model, which then generates optimal renovation ideas for sparsely populated buildings. This AI model proposes the best plan based on pre-trained data.

[0077] Step 8:

[0078] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. This provides users and stakeholders with concrete visual images.

[0079] Step 9:

[0080] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is then formatted to be useful to the user.

[0081] Step 10:

[0082] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[0083] Step 11:

[0084] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[0085] (Example 1)

[0086] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] There is a problem in sparsely populated areas where the utilization of buildings is not progressing, leading to a stagnation of regional revitalization. Furthermore, current renovation plans are not concrete, making it difficult to attract the interest of stakeholders and reach a consensus for implementation. In this situation, there is a need for a system that can quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images to support the realization of renovation plans.

[0088] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0089] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for generating post-renovation design images and videos as promotional materials; means for providing users with concrete visual images to help them plan renovations; and means for providing the generated products as materials for social media and presentations. This makes it possible to quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images.

[0090] "Regional population data" refers to information about the number of people living in a specific area, population density, age distribution, and so on.

[0091] "Traffic data" refers to information regarding traffic volume, the number of traffic accidents, and the types of vehicles traveling in a specific area.

[0092] "Nearby facilities data" refers to information about the location, number, and types of schools, hospitals, supermarkets, public facilities, etc., that exist in a specific area.

[0093] "Human flow data" refers to information that shows the movement patterns and flow of people in a specific area.

[0094] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze, such as imputing missing values, removing outliers, and standardizing the data.

[0095] A "generative AI model" refers to an artificial intelligence model that uses machine learning and deep learning techniques to generate useful information and ideas from new data.

[0096] "Detailed information on sparsely populated buildings from users" refers to information including location information, current usage status, and desired renovations for buildings in sparsely populated areas where renovation is being considered.

[0097] "Renovation ideas" refer to ideas that use generative AI models to suggest optimal ways to utilize or renovate sparsely populated buildings.

[0098] An "image generation model" refers to a model that generates visual design images and videos based on ideas proposed by a generative AI model.

[0099] "Promotional materials" refer to images and videos that visually demonstrate the design and usage of a renovated building, and are intended to attract the attention of project stakeholders and the general public.

[0100] "Visual images" refer to images or videos that visually embody the generated renovation ideas.

[0101] "Materials for social media and presentations" refers to digital files such as images and videos used in social media posts and presentations to promote renovation ideas.

[0102] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each are described below.

[0103] server

[0104] Data Acquisition and Preprocessing

[0105] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data. Specifically, it obtains this data from external APIs (e.g., location APIs and traffic APIs) and databases (e.g., local databases). The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardizing the data. Software tools such as the Python Pandas library are used for preprocessing.

[0106] Training of Generative AI Models

[0107] The server trains a generative AI model using pre-processed data. The generative AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used to train the model, and the data is split into training and validation sets for training.

[0108] Receiving user requests

[0109] The server receives detailed information about sparsely populated buildings entered by the user from their device. This information includes the building's location, current usage, and renovation requests and conditions. This data is sent to the server via an HTTP request.

[0110] Idea generation using AI models

[0111] The server inputs the received detailed information into a generating AI model to generate optimal renovation ideas. Based on the input data, the generating AI model proposes specific ways to utilize the property.

[0112] Creating promotional materials

[0113] The server uses an image generation model to create design images and videos of the completed project based on the generated renovation ideas. This promotional material visually showcases the generated ideas and utilizes image generation technologies such as DeepArt and MidJourney.

[0114] Return of renovation ideas

[0115] The server returns the generated renovation ideas and design images to the user, allowing them to develop a concrete renovation plan.

[0116] terminal

[0117] User Interface

[0118] The terminal provides an interface that allows users to input detailed information about sparsely populated buildings. Through this interface, users can enter details such as the building's location, current photos, and desired renovations.

[0119] Send Request

[0120] The terminal sends the user's entered details to the server. The transmission occurs via an HTTP request, and the data is formatted into the appropriate format.

[0121] Displaying Results

[0122] The terminal displays renovation ideas and design images returned from the server to the user, providing information to help them move forward with concrete project planning.

[0123] User

[0124] Entering information

[0125] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This information includes the building's location, current photos, and their wishes and requirements regarding renovations.

[0126] Idea verification

[0127] Users will view renovation ideas and design images displayed on their devices and use them to create concrete renovation plans. They will also use the generated promotional materials on social media and in presentations to attract the attention of stakeholders.

[0128] Examples of specific cases and prompt statements

[0129] For example, if the user were to use this system to generate renovation ideas for a vacant school building located in a sparsely populated area, they would enter the following prompt:

[0130] "The location of the vacant school building is [latitude and longitude]. Its current use is [description]. The desired use after renovation is a community center. Please generate the most suitable utilization ideas."

[0131] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Then, an image generation model creates design images and videos of the renovated space, and these results are returned to the user's device. The user can then review them and proceed with a concrete renovation plan.

[0132] This will enable the effective support of regional revitalization through a system that generates renovation ideas for sparsely populated buildings using a generation AI model.

[0133] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0134] Step 1: Data Collection

[0135] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. Specifically, the server uses location APIs and traffic APIs to obtain data for a specified area. For example, it obtains local population data from a census API and traffic data from a traffic API. Input requires an API request, and output is data returned in JSON format.

[0136] Step 2: Data Preprocessing

[0137] The server preprocesses the collected data. Specifically, it uses the Python Pandas library to impute missing values, remove outliers, and standardize the data. The data collected in step 1 is used as input, and the output is clean, preprocessed data. This is to prepare the data for training the generative AI model.

[0138] Step 3: Training the Generative AI Model

[0139] The server uses preprocessed data to train a generative AI model. Specifically, it trains a deep learning model using TensorFlow or PyTorch. Preprocessed data is used as input, and the trained generative AI model is obtained as output. Furthermore, the model is evaluated at this stage to confirm its performance.

[0140] Step 4: Receiving User Requests

[0141] The terminal sends detailed information about sparsely populated buildings entered by the user to the server. Specifically, it sends information obtained through the user interface as an HTTP request. The input is information entered by the user (location information, current photos, renovation requests), and the output is received by the server.

[0142] Step 5: Idea generation using AI models

[0143] The server inputs the received user details into a generating AI model to generate optimal renovation ideas. Specifically, the generating AI model proposes utilization methods based on the input data. The input uses detailed information received from the user, and the output is the generated renovation ideas.

[0144] Step 6: Create promotional materials

[0145] The server uses an image generation model to create design images and videos based on the generated renovation ideas. Specifically, it utilizes tools such as DeepArt and MidJourney to generate visuals of the renovated area. The generated renovation ideas are used as input, and promotional materials in image and video formats are obtained as output.

[0146] Step 7: Return of renovation ideas

[0147] The server returns the generated renovation ideas and promotional materials to the terminal. Specifically, it formats the generated content and sends it to the terminal as an HTTP response. The generated content (ideas and materials) is used as input, and the output is sent back to the terminal.

[0148] Step 8: Displaying the results

[0149] The terminal displays renovation ideas and promotional materials returned from the server to the user. Specifically, it displays the returned data on the user interface and formats it so that the user can review it. The input is the data returned from the server, and the output is displayed in a format that the user can view. The user then uses this to create a concrete renovation plan.

[0150] (Application Example 1)

[0151] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0152] Renovating buildings in sparsely populated areas is difficult due to population decline and decreased demand. Furthermore, the process of proposing concrete renovation ideas and designs is cumbersome, highlighting the need for effective methods of regional revitalization. Therefore, a system is needed to quickly and efficiently generate and provide renovation ideas and concrete designs for physical stores in sparsely populated areas.

[0153] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0154] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; and means for learning a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data. This makes it possible to quickly and efficiently generate renovation ideas for physical stores located in sparsely populated areas and provide users with design images and videos based on those ideas.

[0155] The server also includes means for receiving detailed information about sparsely populated buildings from users, means for inputting the received detailed information into a generation AI model to generate renovation ideas, means for generating completed design images and videos using an image generation model based on the generated ideas, and means for returning the generated renovation ideas and design images and videos to the user via a smartphone application. This allows users to easily check specific renovation ideas and their visual images, and use them to plan for regional revitalization.

[0156] "Regional population data" refers to information about the population size, population density, age distribution, etc., within a specific region.

[0157] "Traffic data" refers to information regarding traffic volume, congestion, and the operating status of public transportation within a specific area.

[0158] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, entertainment facilities, and other facilities located within a specific area.

[0159] "People flow data" refers to information about the movement and dwell time of people within a specific area, as well as the number of visitors and their travel routes.

[0160] "Preprocessing" refers to the process of preparing collected data for training AI models by performing actions such as imputing missing values, removing outliers, and standardizing the data.

[0161] A "generative AI model" is an artificial intelligence model that learns from collected and pre-processed data to generate optimal utilization ideas for sparsely populated buildings.

[0162] "Detailed information on underutilized buildings from users" refers to specific information such as the location of the underutilized building, its current use, size, photos, and any wishes or conditions regarding renovation.

[0163] An "image generation model" is an artificial intelligence model that generates design images and videos of a renovated space based on ideas generated by a generative AI model.

[0164] A "smartphone application" is application software that runs on a smartphone and allows users to input detailed information about renovations and view renovation ideas, design images, and videos returned from a server.

[0165] "Promotional materials" refer to the generated renovation ideas and the design images and videos based on them, which will be used for promotional activities aimed at revitalizing the local community.

[0166] This invention is a system that utilizes generative AI models to renovate buildings in sparsely populated areas and support regional revitalization. The system is divided into three main components: a server, terminals, and users.

[0167] server

[0168] Data Acquisition and Preprocessing

[0169] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This data is preprocessed, including imputing missing values, removing outliers, and standardization, to prepare it as training data. Data processing libraries such as Python and Pandas are used for collection and preprocessing.

[0170] Training of Generative AI Models

[0171] The server trains a generative AI model using preprocessed data. Deep learning frameworks such as TensorFlow and PyTorch are used to train the AI ​​model. This AI model generates optimal utilization ideas for sparsely populated buildings. During the training process, the data is divided into training data and validation data, and the model's performance is evaluated.

[0172] Processing the request

[0173] The server receives detailed information about sparsely populated buildings sent by users via their devices. This includes the building's location, current usage, and any wishes or conditions for renovation. The server inputs the received information into an AI model to generate renovation ideas.

[0174] Idea generation and promotional material creation

[0175] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. Technologies such as Generative Opposite Networks (GANs) are used for image generation. The generated renovation ideas and design images and videos are returned to the user via a smartphone application.

[0176] terminal

[0177] User Interface

[0178] The terminal provides users with an interface for inputting detailed information about sparsely populated buildings. Users can use the terminal to input information such as the building's location, current photos, and renovation requests. This allows users to easily send the necessary information to the server.

[0179] Send Request

[0180] The terminal formats the information entered by the user into a data format and sends it to the server. The HTTPS protocol is used to send requests, ensuring secure and efficient data exchange.

[0181] Displaying Results

[0182] The server returns renovation ideas and promotional materials (design images and videos) to the user. This allows the user to create a concrete renovation plan.

[0183] User

[0184] Entering information

[0185] Users input detailed information about sparsely populated buildings they are interested in through the interface of a smartphone application. This includes, for example, the building's location, size, current use, photos, and their wishes and requirements for renovation.

[0186] Idea verification

[0187] Users can review renovation ideas and promotional materials displayed on their devices. Based on the suggested ideas, users can proceed with developing concrete project plans. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0188] Specific example

[0189] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information, current condition (with photos), and desired use (e.g., to use it as a community center) of the vacant school building through a smartphone application interface. The server receives this information and uses a generation AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated building. These results are returned to the user via the smartphone application, allowing the user to review them and develop a concrete renovation plan.

[0190] Example of a prompt

[0191] "This building is currently vacant. We would like to renovate it and turn it into a shop selling local specialty products. Please generate a renovation plan for the building, create an image rendering and a promotional video. We have attached some reference photos."

[0192] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[0193] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0194] Step 1:

[0195] Data collection (server)

[0196] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed using the Python requests library, and database management systems such as SQLite and PostgreSQL are used for queries from the databases. Inputs are external API endpoints and database queries, and outputs are the respective data.

[0197] Step 2:

[0198] Data preprocessing (server)

[0199] The collected data is preprocessed. This preprocessing includes imputing missing values, removing outliers, and standardization. The program uses the Pandas library to perform data cleaning. Specifically, missing values ​​are imputed with the median, and outliers (e.g., data exceeding ±3σ) are removed. The input is the collected raw data, and the output is the preprocessed data.

[0200] Step 3:

[0201] Training of generative AI models (server)

[0202] The server trains a generative AI model using preprocessed data. It builds and trains the model using deep learning frameworks such as TensorFlow and PyTorch. The data is divided into training and validation data, and the model's performance is evaluated throughout the training process. The input is preprocessed data, and the output is the trained generative AI model.

[0203] Step 4:

[0204] Receiving user input information (terminal)

[0205] The terminal provides a user interface, allowing users to input detailed information about sparsely populated buildings. This information includes the building's location, current use, size, photos, and any desired renovations or requirements. The input is the building information entered by the user, and the output is the data organized for transmission to the server.

[0206] Step 5:

[0207] Sending a request (from the terminal)

[0208] The terminal organizes the information entered by the user and generates a request to send to the server. The request contains all the collected information in JSON format. The input is the building information entered by the user, and the output is the request data sent to the server.

[0209] Step 6:

[0210] Renovation idea generation (server)

[0211] The server inputs detailed information received from the user into a generating AI model to generate renovation ideas. During this process, prompt statements are created and input into the model. The input is the user's building information, and the output is the generated renovation ideas.

[0212] Step 7:

[0213] Design image and video generation (server)

[0214] Based on the generated renovation ideas, the server uses an image generation model to produce images and videos of the completed design. This process employs generative opposite-party network (GAN) technology. The input is the renovation idea, and the output is the generated images and videos.

[0215] Step 8:

[0216] Return of results (server)

[0217] The generated renovation ideas, design images, and videos are returned to the user via a smartphone application. The input is design images and videos, and the output is the result displayed on the user's device.

[0218] Step 9:

[0219] Checking the results (user)

[0220] Users view renovation ideas and promotional materials (images and videos) displayed on their devices. Input is information returned from the server, and output is the visual content the user reviews. This allows for the creation of concrete renovation plans.

[0221] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0222] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[0223] server

[0224] Data Acquisition and Preprocessing

[0225] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. The collected data is preprocessed, including imputation of missing values, removal of outliers, and standardization, to prepare it for training AI models.

[0226] Training of Generative AI Models

[0227] The server uses pre-processed data to train a generative AI model that generates optimal utilization ideas for sparsely populated buildings. Using a deep learning framework (e.g., TensorFlow or PyTorch), the model is trained on training data and validation data, and its performance is evaluated.

[0228] Processing the request

[0229] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. The received information includes the building's location, current usage, and the user's wishes and requirements. The server inputs this detailed information into a generating AI model to generate renovation ideas.

[0230] Emotional engine integration

[0231] The emotion engine analyzes the user's emotional data and works in conjunction with a generative AI model to optimize renovation idea recommendations. Based on the information entered by the user and the emotional data analyzed by the emotion engine, more personalized renovation ideas are generated.

[0232] Idea generation and promotional material creation

[0233] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated ideas and promotional materials are then returned to the terminal.

[0234] terminal

[0235] User Interface

[0236] The device provides users with an interface for inputting detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, renovation wishes and conditions, and even their own feelings.

[0237] Send Request

[0238] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[0239] Displaying Results

[0240] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[0241] User

[0242] Entering information

[0243] Through the terminal interface, users can input detailed information and emotional data about sparsely populated buildings that interest them. For example, they can input the building's location, size, and current use, as well as their wishes and conditions regarding renovations, and their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[0244] Idea verification

[0245] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0246] Specific example

[0247] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. Through the terminal interface, the user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project).

[0248] The server receives this information and uses a generative AI model to generate optimal renovation ideas. The emotion engine analyzes the user's emotional data and reflects the results in the generative AI model, providing personalized ideas that take the user's emotions into consideration. Based on the generated ideas, the image generation model creates design images and videos of the renovated space. These results are returned to the device, allowing the user to review them and develop a concrete plan.

[0249] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[0250] The following describes the processing flow.

[0251] Step 1:

[0252] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This allows for accurate and up-to-date information about sparsely populated areas.

[0253] Step 2:

[0254] The server preprocesses the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. In this way, it prepares the data in a format suitable for training.

[0255] Step 3:

[0256] The server uses pre-processed data to train a generative AI model. A deep learning framework (such as TensorFlow or PyTorch) is used for training, dividing the data into training and validation sets to train the model, and then its performance is evaluated.

[0257] Step 4:

[0258] Users input detailed information about sparsely populated buildings through the terminal's user interface. This information includes location data, current photos, renovation wishes and requirements, and even emotional data (e.g., excitement, anticipation, anxiety).

[0259] Step 5:

[0260] The terminal formats the information entered by the user and sends it to the server. The terminal performs formatting and error checking to ensure that the data is transmitted accurately.

[0261] Step 6:

[0262] The server receives detailed information and sentiment data about sparsely populated buildings sent from the terminal. The received information includes the building's location and details about its usage.

[0263] Step 7:

[0264] The server inputs the received detailed information and emotional data into the generating AI model and emotion engine to generate optimal renovation ideas for sparsely populated buildings. In this process, the emotion engine analyzes the user's emotions and reflects the results in the generating AI model.

[0265] Step 8:

[0266] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. The generated designs are mindful of the user's emotions.

[0267] Step 9:

[0268] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is formatted into a user-friendly format.

[0269] Step 10:

[0270] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[0271] Step 11:

[0272] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[0273] (Example 2)

[0274] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0275] In the renovation of buildings in sparsely populated areas, conventional methods failed to effectively utilize region-specific data, and it was difficult to provide personalized proposals that reflected user sentiment. This made it difficult to generate appropriate utilization ideas and to realize concrete plans that would contribute to regional revitalization. In particular, proposals that did not consider user sentiment were problematic because they made it difficult to motivate actual initiatives and gain the support of stakeholders.

[0276] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0277] In this invention, the server includes means for collecting regional population data, traffic situation data, neighboring facility data, and pedestrian flow data, means for preprocessing the collected data, means for training a generative AI model that uses the preprocessed data to generate optimal utilization ideas for underpopulated buildings, means for receiving detailed information on underpopulated buildings from the user, means for inputting the received detailed information into the generative AI model to generate renovation ideas, means for generating a completed design image using an image generation model based on the generated ideas, means for returning the generated renovation ideas and design images to the user, means for analyzing the user's emotional data, and means for reflecting the analyzed emotional data in the generative AI model. This enables the generation of personalized renovation ideas that reflect regional-specific data and the user's emotions.

[0278] The "regional population data" refers to information about the population distribution and population dynamics in a specific region.

[0279] The "traffic situation data" refers to information regarding the traffic flow, traffic congestion situation, traffic volume, etc. in a specific region.

[0280] The "neighboring facility data" refers to information about the locations and characteristics of public facilities, commercial facilities, educational facilities, etc. existing within a specific region.

[0281] The "pedestrian flow data" refers to information regarding the movement patterns and flow of people in a specific region.

[0282] "Preprocessing" refers to processes such as filling in missing values, removing outliers, and normalizing data to prepare it in a form suitable for data analysis and model training.

[0283] The "generative AI model" is a model for generating a target output from specific input data using artificial intelligence technology.

[0284] <000​

[0285] "Training a generative AI model" means performing a training process to adjust the parameters of a generative AI model using appropriate input data to obtain high-precision output.

[0286] "User sentiment data" refers to information regarding the emotions felt by the user, such as data on various emotions like expectation, anxiety, and joy.

[0287] "Sentiment engine" refers to a system that analyzes the emotions from the data input by the user and outputs them in a specific format.

[0288] "Image generation model" refers to an artificial intelligence model for generating visual materials such as images and videos based on specific input data.

[0289] "User interface" refers to the operating means such as screens and forms used by the user to input information into the system.

[0290] "Promotion materials" refer to visual and text content used to promote projects and proposals using generated renovation ideas, design images, etc.

[0291] The present invention combines a sentiment engine that recognizes the emotions of users with a system that utilizes generative AI to renovate buildings in depopulated areas and support regional revitalization. The system is composed of three main components: a server, a terminal, and a user, and their respective roles and specific processes are as follows.

[0292] Server processing

[0293] Data collection and preprocessing

[0294] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from statistical agency APIs, and traffic data is obtained from traffic information services. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. Methods such as imputing missing values ​​with the mean or mode are used. Outlier removal is performed by deleting data that is statistically determined to be outliers. Finally, the data is standardized to shape it into a form suitable for model training.

[0295] Training of Generative AI Models

[0296] The server uses pre-processed data to train a generative AI model to generate optimal utilization ideas for sparsely populated buildings. The deep learning frameworks used include TensorFlow and PyTorch. During training, the dataset is divided into training and validation data, and the model's performance is evaluated and optimized. For example, 80% of the data might be used as training data and 20% as validation data.

[0297] Processing the request

[0298] The server receives detailed information about sparsely populated buildings sent from users via their devices. This information includes the building's location, current usage, and the user's wishes and requirements. This information is then input into an AI model to generate renovation ideas. For example, if the target is a closed school in XX City, Nagano Prefecture, the server would provide information such as its location, current usage, and the user's wishes (e.g., wanting to use it as a community center).

[0299] Emotional engine integration

[0300] The server analyzes the user's emotional data using an emotion engine. The emotion engine extracts the user's emotions from the text using NLP technology and reflects the data in the generated AI model. For example, when the user inputs "I have expectations for this project, but I also have some anxiety", the emotions of "expectation" and "anxiety" are analyzed.

[0301] Idea generation and promotion material creation

[0302] Based on the generated renovation ideas, the server uses an image generation model to generate design images and videos after completion. For example, a GAN (Generative Adversarial Network) is used to generate realistic design images. The generated ideas and design images are used as promotion materials and returned to the terminal.

[0303] Terminal processing

[0304] User interface

[0305] The terminal provides an interface for the user to input detailed information about the underpopulated building. For example, there are functions such as forms and photo uploads. The user uses this interface to input location information, current photos, desired usage methods, and emotions.

[0306] Request sending

[0307] The terminal formats the information input by the user and sends it to the server. For example, the data is converted into JSON format and sent after error checking.

[0308] Result display

[0309] The renovation ideas and promotion materials (design images and videos) returned from the server are displayed to the user. The terminal provides a viewer for displaying the generated design images and videos so that the user can easily check them.

[0310] User processing

[0311] Entering information

[0312] Through the terminal interface, users input information about the building's location, size, current use, and their wishes, conditions, and emotions (excitement, anticipation, anxiety, etc.) regarding the renovation. For example, they might input their wish to "use a closed school in XX City, Nagano Prefecture as a community center," along with photos of the current state and their "expectations and anxieties about the project."

[0313] Idea verification

[0314] Review the proposed renovation ideas and promotional materials. View the generated design images and videos to develop a concrete renovation plan. For example, determine the project's progress based on the design images and videos displayed on the device.

[0315] Example of a prompt

[0316] "We want to use a closed school located in a sparsely populated area as a community center. Please generate renovation ideas based on the following information:

[0317] Building location: Nagano Prefecture, XX City, △△ Town

[0318] Current photos: xxx.jpg, yyy.jpg

[0319] Current use: A closed elementary school

[0320] User emotions: Expectation (want to contribute to regional revitalization), anxiety (worried about whether it will succeed)

[0321] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. Furthermore, the introduction of an emotion engine enables personalized suggestions that take into account the user's emotions.

[0322] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0323] Step 1: Data Collection

[0324] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, it uses APIs from statistical agencies and traffic information services.

[0325] Input: Raw data from external APIs or databases.

[0326] Output: Collected raw data.

[0327] Step 2: Data Preprocessing

[0328] The server performs missing value imputation, outlier removal, and standardization on the collected data. For example, it imputates missing values ​​with the mean, statistically removes outliers, and normalizes numerical data from 0 to 1.

[0329] Input: Collected raw data.

[0330] Output: Pre-processed data.

[0331] Step 3: Training the Generative AI Model

[0332] The server uses pre-processed data to train a generative AI model. For example, it can use TensorFlow or PyTorch to prepare training and validation data and then train the model.

[0333] Input: Pre-processed data.

[0334] Output: A trained generative AI model.

[0335] Step 4: Receiving User Information

[0336] The server receives detailed information about sparsely populated buildings sent from users via their terminals. This includes, for example, the building's location, current usage, and the user's preferences and requirements.

[0337] Input: Building information submitted by the user.

[0338] Output: Detailed information stored on the server.

[0339] Step 5: Generating renovation ideas

[0340] The server inputs the received detailed information into a generating AI model to produce renovation ideas. For example, it proposes the optimal renovation plan based on the building's location and intended use.

[0341] Input: Detailed information and a trained generative AI model.

[0342] Output: Renovation ideas.

[0343] Step 6: Analyzing emotional data

[0344] The server analyzes emotional data from users using an emotion engine. For example, it extracts emotions from text using NLP (Neuro-Linguistic Programming) techniques.

[0345] Input: User sentiment data.

[0346] Output: Analyzed sentiment data.

[0347] Step 7: Personalized Idea Generation

[0348] The server incorporates the analyzed emotional data into a generating AI model to create personalized ideas. It provides renovation ideas that reflect the user's emotions.

[0349] Input: Analyzed sentiment data and detailed information.

[0350] Output: Personalized renovation ideas.

[0351] Step 8: Create promotional materials

[0352] The server uses image generation models to create design images and videos based on the generated renovation ideas. For example, it uses GANs to generate realistic images of the finished product.

[0353] Input: Generated renovation ideas.

[0354] Output: Promotional materials (design images, videos).

[0355] Step 9: Return of Results

[0356] The server returns the generated renovation ideas and promotional materials to the terminal and provides them to the user.

[0357] Input: Promotional material.

[0358] Output: Data returned to the terminal.

[0359] Step 10: User Input

[0360] Users input detailed information such as the building's location, photos, desired usage, and emotions through the device's interface.

[0361] Input: Various data entered by the user.

[0362] Output: Detailed data sent to the terminal.

[0363] Step 11: Submit Request

[0364] The terminal formats the information entered by the user and sends it to the server. For example, it converts the information to JSON format, performs error checking, and then sends it.

[0365] Input: Detailed information entered by the user.

[0366] Output: Request data sent to the server.

[0367] Step 12: Displaying the results

[0368] The terminal displays renovation ideas and promotional materials returned from the server to the user. For example, it provides a viewer that allows users to view generated design images and videos.

[0369] Input: Data returned from the server.

[0370] Output: Data displayed to the user.

[0371] (Application Example 2)

[0372] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0373] It is crucial to effectively renovate buildings in sparsely populated areas and promote regional revitalization. However, what is needed is not just the generation of renovation ideas, but also personalized proposals that take into account the user's feelings. Furthermore, there is a lack of visual means to convey how the generated ideas can be specifically useful. To solve these problems, a system is needed that proposes the optimal utilization methods for sparsely populated buildings, as well as providing optimal renovation ideas and promotional materials based on the user's feelings.

[0374] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0375] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; and means including an emotion engine that analyzes emotion data and works in conjunction with the generative AI model to optimize renovation ideas based on the user's emotions. This makes it possible to provide personalized renovation ideas and concrete visual images that meet the user's wishes and emotions.

[0376] "Regional population data" refers to statistical information such as the number of people residing in a specific area, the number of households, age groups, and gender.

[0377] "Traffic data" refers to information such as traffic volume, congestion levels, and the operating status of public transportation in a specific area.

[0378] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, public facilities, and other similar facilities located in a specific area.

[0379] "People flow data" refers to information such as people's movement patterns, flow conditions, and ability to attract customers in a specific area.

[0380] "Preprocessing" refers to the data organization and editing work required to apply large amounts of data to machine learning models.

[0381] A "generative AI model" refers to an artificial intelligence model that uses machine learning to generate optimal renovation ideas from data.

[0382] "Detailed information" refers to information provided by users, such as the location, size, current usage, and desired conditions or requirements for renovations of sparsely populated buildings.

[0383] "Emotional data" refers to information about a user's psychological state and emotions (e.g., enjoyment, anticipation, anxiety) obtained from user input, facial expressions, voice, and behavior.

[0384] An "emotion engine" refers to a system that analyzes user emotional data and incorporates the results into the generated AI model.

[0385] An "image generation model" refers to a model that uses AI to generate actual images from text and other data.

[0386] "Renovation ideas" refer to specific proposals and design plans for how to remodel and reuse sparsely populated buildings.

[0387] "Personalized suggestions" refer to providing individual renovation ideas optimized based on each user's wishes and feelings.

[0388] "Promotional materials" refer to advertising and public relations content created using generated renovation ideas and design images.

[0389] This invention is a system that generates and provides renovation ideas for buildings in sparsely populated areas, thereby supporting regional revitalization. This system consists of the following three main components:

[0390] server

[0391] The server first collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from the census database, traffic data from the traffic management system, nearby facilities data from the Geographic Information System (GIS), and pedestrian flow data from the mobile data analytics service.

[0392] The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardization. For data preprocessing, the Python pandas library is used, for example.

[0393] Using pre-processed data, the server trains a generative AI model using a deep learning framework (TensorFlow or PyTorch). This model is used to generate optimal utilization ideas for sparsely populated buildings. This allows the model to learn diverse data patterns and produce highly accurate generation results.

[0394] When a user sends detailed information about a sparsely populated building from their device, the server receives this information and inputs it into the generated AI model. This detailed information includes the building's location, current usage, and the user's wishes and conditions regarding renovations.

[0395] Furthermore, the server uses an emotion engine to analyze the user's emotional data and integrates this result with a generating AI model to create more personalized renovation ideas. Based on the analyzed emotional data, the server makes suggestions that take into account the user's expectations and anxieties.

[0396] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the finished product. This allows users to have a concrete visual image of the project. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream.

[0397] terminal

[0398] The terminal provides users with a user interface for entering detailed information about sparsely populated buildings. Through this interface, users can input the building's location, current photos, and their wishes and requirements for renovation. Furthermore, they can also input their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[0399] The information entered by the user is formatted into the appropriate format by the terminal and sent to the server. The terminal also performs error checking on the information to ensure that the data is transmitted accurately.

[0400] Renovation ideas and promotional materials (images and videos) returned from the server are displayed on the device. This allows users to create concrete renovation plans based on these materials.

[0401] User

[0402] Users input detailed information and sentimental data about sparsely populated buildings they are interested in via their devices. For example, they input the building's location, size, and current use, as well as their wishes, conditions, and feelings regarding renovations (e.g., excitement, anticipation, anxiety).

[0403] Users can view renovation ideas and promotional materials displayed on their devices and use them to develop concrete project plans. The generated materials can also be used on social media and in presentations, serving as valuable tools to attract the attention of local communities and stakeholders.

[0404] Specific example

[0405] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. The user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project) from their device.

[0406] The server receives this information and generates renovation ideas using a generative AI model. The emotion engine analyzes the user's emotional data and incorporates the results into the generative AI model, providing personalized ideas that take the user's emotions into consideration.

[0407] Example of a prompt

[0408] User input:

[0409] Building details: Abandoned school in Nagano Prefecture, area 1200m² 2 Currently used as a warehouse.

[0410] Renovation request: I would like to use it as a community center.

[0411] Emotions: I'm very excited, but also a little anxious.

[0412] Expected output:

[0413] Generated from AI model: "By renovating the abandoned school into a community center, local interaction can be deepened and it can be used as a new tourist attraction. In particular, it is important to incorporate a design that prioritizes comfort. Since users have given positive feedback, let's give maximum consideration to comfort and functionality."

[0414] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[0415] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0416] Step 1:

[0417] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. For example, population data is obtained from a census database, and traffic data is obtained from a traffic management system. The input is raw data obtained from APIs and databases, and the output is preprocessed, formatted data.

[0418] Step 2:

[0419] The server uses preprocessed data to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). This allows the server to build a generative AI model for generating optimal utilization ideas for sparsely populated buildings. The input is preprocessed data, and the output is the trained generative AI model. Specifically, the model is trained using training data, and its performance is evaluated.

[0420] Step 3:

[0421] Users input detailed information about sparsely populated buildings and their own feelings using a user interface on their device. This might include the building's location, current photos, renovation hopes and conditions, and emotions (e.g., excitement, anticipation, anxiety). The input is information provided by the user, and the output is formatted detailed data.

[0422] Step 4:

[0423] The terminal formats the user-entered details into the appropriate format and sends them to the server. It also performs error checking to ensure the data is sent accurately to the server. The input is the information entered by the user, and the output is the formatted data after error checking.

[0424] Step 5:

[0425] The server receives detailed information sent by the user and inputs this information into the generating AI model. Specifically, it supplies the AI ​​model with information such as the building's location, usage, and the user's wishes and conditions. The input is the detailed information received from the user, and the output is the generated renovation ideas.

[0426] Step 6:

[0427] The server analyzes emotional data and works with a generative AI model to optimize renovation ideas based on the user's emotions. The emotion engine analyzes user input, facial expressions, voice, etc., and provides emotion-based data. The input is the user's emotional data, and the output is an optimized renovation idea based on those emotions.

[0428] Step 7:

[0429] The server uses an image generation model to generate images and videos of the completed design based on the generated renovation ideas. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream. The input is the generated renovation idea, and the output is the images and videos of the completed design.

[0430] Step 8:

[0431] The server returns the generated renovation ideas and design images to the terminal. This allows the user to have a concrete visual image. The input is the generated renovation ideas and design images, and the output is the data returned to the user.

[0432] Step 9:

[0433] The terminal displays renovation ideas and promotional materials (images and videos) returned from the server to the user. This allows the user to create a concrete renovation plan based on these materials. The input is data returned from the server, and the output is the renovation ideas and design images displayed to the user.

[0434] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0435] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0436] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0437] [Second Embodiment]

[0438] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0439] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0440] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0441] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0442] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0443] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0444] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0445] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0446] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0447] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0448] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0449] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0450] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[0451] server

[0452] Data Acquisition and Preprocessing

[0453] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. It then performs preprocessing on the collected data, such as imputing missing values, removing outliers, and standardizing the data. This prepares the data for training.

[0454] Training of Generative AI Models

[0455] The server trains a generative AI model using preprocessed data. This AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used for training, and the data is divided into training and validation data for model training. Model evaluation is also performed at this stage to verify its performance.

[0456] Processing the request

[0457] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. This information includes the building's location and current usage. The server inputs this information into an AI model to generate ideas for its utilization.

[0458] Idea generation and promotional material creation

[0459] Based on the generated utilization ideas, the server uses an image generation model to create images and videos that visually represent the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated utilization ideas and promotional materials are then returned to the terminal.

[0460] terminal

[0461] User Interface

[0462] The terminal provides users with an interface for entering detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, and their wishes and conditions regarding renovations.

[0463] Send Request

[0464] To send the information entered by the user to the server, the terminal formats the data and sends the request.

[0465] Displaying Results

[0466] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[0467] User

[0468] Entering information

[0469] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This includes, for example, the building's location, size, and current use. Users can also enter their wishes and conditions regarding renovations.

[0470] Idea verification

[0471] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0472] Specific example

[0473] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information of the vacant school building, its current condition (with photos), and their desired use (for example, to use it as a community center) through the terminal interface.

[0474] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated space. These results are returned to the terminal, allowing the user to review them and develop a concrete renovation plan.

[0475] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[0476] The following describes the processing flow.

[0477] Step 1:

[0478] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed regularly with the aim of obtaining accurate and up-to-date information.

[0479] Step 2:

[0480] The server performs preprocessing on the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. This prepares a dataset suitable for subsequent AI model training.

[0481] Step 3:

[0482] The server trains a generative AI model using preprocessed data. It splits the data into training data and validation data, trains the model using a deep learning framework (e.g., TensorFlow or PyTorch), and evaluates the model's performance.

[0483] Step 4:

[0484] Users input detailed information about sparsely populated buildings (location, current photos, renovation requests and conditions, etc.) through the terminal's user interface. Once the input is complete, they submit a request for renovation idea generation.

[0485] Step 5:

[0486] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[0487] Step 6:

[0488] The server receives detailed information about sparsely populated buildings sent from the terminal. The received information includes the building's location and details on how to use it.

[0489] Step 7:

[0490] The server inputs the received information into a generating AI model, which then generates optimal renovation ideas for sparsely populated buildings. This AI model proposes the best plan based on pre-trained data.

[0491] Step 8:

[0492] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. This provides users and stakeholders with concrete visual images.

[0493] Step 9:

[0494] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is then formatted to be useful to the user.

[0495] Step 10:

[0496] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[0497] Step 11:

[0498] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[0499] (Example 1)

[0500] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0501] There is a problem in sparsely populated areas where the utilization of buildings is not progressing, leading to a stagnation of regional revitalization. Furthermore, current renovation plans are not concrete, making it difficult to attract the interest of stakeholders and reach a consensus for implementation. In this situation, there is a need for a system that can quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images to support the realization of renovation plans.

[0502] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0503] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for generating post-renovation design images and videos as promotional materials; means for providing users with concrete visual images to help them plan renovations; and means for providing the generated products as materials for social media and presentations. This makes it possible to quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images.

[0504] "Regional population data" refers to information about the number of people living in a specific area, population density, age distribution, and so on.

[0505] "Traffic data" refers to information regarding traffic volume, the number of traffic accidents, and the types of vehicles traveling in a specific area.

[0506] "Nearby facilities data" refers to information about the location, number, and types of schools, hospitals, supermarkets, public facilities, etc., that exist in a specific area.

[0507] "Human flow data" refers to information that shows the movement patterns and flow of people in a specific area.

[0508] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze, such as imputing missing values, removing outliers, and standardizing the data.

[0509] A "generative AI model" refers to an artificial intelligence model that uses machine learning and deep learning techniques to generate useful information and ideas from new data.

[0510] "Detailed information on sparsely populated buildings from users" refers to information including location information, current usage status, and desired renovations for buildings in sparsely populated areas where renovation is being considered.

[0511] "Renovation ideas" refer to ideas that use generative AI models to suggest optimal ways to utilize or renovate sparsely populated buildings.

[0512] An "image generation model" refers to a model that generates visual design images and videos based on ideas proposed by a generative AI model.

[0513] "Promotional materials" refer to images and videos that visually demonstrate the design and usage of a renovated building, and are intended to attract the attention of project stakeholders and the general public.

[0514] "Visual images" refer to images or videos that visually embody the generated renovation ideas.

[0515] "Materials for social media and presentations" refers to digital files such as images and videos used in social media posts and presentations to promote renovation ideas.

[0516] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each are described below.

[0517] server

[0518] Data Acquisition and Preprocessing

[0519] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data. Specifically, it obtains this data from external APIs (e.g., location APIs and traffic APIs) and databases (e.g., local databases). The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardizing the data. Software tools such as the Python Pandas library are used for preprocessing.

[0520] Training of Generative AI Models

[0521] The server trains a generative AI model using pre-processed data. The generative AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used to train the model, and the data is split into training and validation sets for training.

[0522] Receiving user requests

[0523] The server receives detailed information about sparsely populated buildings entered by the user from their device. This information includes the building's location, current usage, and renovation requests and conditions. This data is sent to the server via an HTTP request.

[0524] Idea generation using AI models

[0525] The server inputs the received detailed information into a generating AI model to generate optimal renovation ideas. Based on the input data, the generating AI model proposes specific ways to utilize the property.

[0526] Creating promotional materials

[0527] The server uses an image generation model to create design images and videos of the completed project based on the generated renovation ideas. This promotional material visually showcases the generated ideas and utilizes image generation technologies such as DeepArt and MidJourney.

[0528] Return of renovation ideas

[0529] The server returns the generated renovation ideas and design images to the user, allowing them to develop a concrete renovation plan.

[0530] terminal

[0531] User Interface

[0532] The terminal provides an interface that allows users to input detailed information about sparsely populated buildings. Through this interface, users can enter details such as the building's location, current photos, and desired renovations.

[0533] Send Request

[0534] The terminal sends the user's entered details to the server. The transmission occurs via an HTTP request, and the data is formatted into the appropriate format.

[0535] Displaying Results

[0536] The terminal displays renovation ideas and design images returned from the server to the user, providing information to help them move forward with concrete project planning.

[0537] User

[0538] Entering information

[0539] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This information includes the building's location, current photos, and their wishes and requirements regarding renovations.

[0540] Idea verification

[0541] Users will view renovation ideas and design images displayed on their devices and use them to create concrete renovation plans. They will also use the generated promotional materials on social media and in presentations to attract the attention of stakeholders.

[0542] Examples of specific cases and prompt statements

[0543] For example, if the user were to use this system to generate renovation ideas for a vacant school building located in a sparsely populated area, they would enter the following prompt:

[0544] "The location of the vacant school building is [latitude and longitude]. Its current use is [description]. The desired use after renovation is a community center. Please generate the most suitable utilization ideas."

[0545] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Then, an image generation model creates design images and videos of the renovated space, and these results are returned to the user's device. The user can then review them and proceed with a concrete renovation plan.

[0546] This will enable the effective support of regional revitalization through a system that generates renovation ideas for sparsely populated buildings using a generation AI model.

[0547] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0548] Step 1: Data Collection

[0549] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. Specifically, the server uses location APIs and traffic APIs to obtain data for a specified area. For example, it obtains local population data from a census API and traffic data from a traffic API. Input requires an API request, and output is data returned in JSON format.

[0550] Step 2: Data Preprocessing

[0551] The server preprocesses the collected data. Specifically, it uses the Python Pandas library to impute missing values, remove outliers, and standardize the data. The data collected in step 1 is used as input, and the output is clean, preprocessed data. This is to prepare the data for training the generative AI model.

[0552] Step 3: Training the Generative AI Model

[0553] The server uses preprocessed data to train a generative AI model. Specifically, it trains a deep learning model using TensorFlow or PyTorch. Preprocessed data is used as input, and the trained generative AI model is obtained as output. Furthermore, the model is evaluated at this stage to confirm its performance.

[0554] Step 4: Receiving User Requests

[0555] The terminal sends detailed information about sparsely populated buildings entered by the user to the server. Specifically, it sends information obtained through the user interface as an HTTP request. The input is information entered by the user (location information, current photos, renovation requests), and the output is received by the server.

[0556] Step 5: Idea generation using AI models

[0557] The server inputs the received user details into a generating AI model to generate optimal renovation ideas. Specifically, the generating AI model proposes utilization methods based on the input data. The input uses detailed information received from the user, and the output is the generated renovation ideas.

[0558] Step 6: Create promotional materials

[0559] The server uses an image generation model to create design images and videos based on the generated renovation ideas. Specifically, it utilizes tools such as DeepArt and MidJourney to generate visuals of the renovated area. The generated renovation ideas are used as input, and promotional materials in image and video formats are obtained as output.

[0560] Step 7: Return of renovation ideas

[0561] The server returns the generated renovation ideas and promotional materials to the terminal. Specifically, it formats the generated content and sends it to the terminal as an HTTP response. The generated content (ideas and materials) is used as input, and the output is sent back to the terminal.

[0562] Step 8: Displaying the results

[0563] The terminal displays renovation ideas and promotional materials returned from the server to the user. Specifically, it displays the returned data on the user interface and formats it so that the user can review it. The input is the data returned from the server, and the output is displayed in a format that the user can view. The user then uses this to create a concrete renovation plan.

[0564] (Application Example 1)

[0565] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0566] Renovating buildings in sparsely populated areas is difficult due to population decline and decreased demand. Furthermore, the process of proposing concrete renovation ideas and designs is cumbersome, highlighting the need for effective methods of regional revitalization. Therefore, a system is needed to quickly and efficiently generate and provide renovation ideas and concrete designs for physical stores in sparsely populated areas.

[0567] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0568] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; and means for learning a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data. This makes it possible to quickly and efficiently generate renovation ideas for physical stores located in sparsely populated areas and provide users with design images and videos based on those ideas.

[0569] The server also includes means for receiving detailed information about sparsely populated buildings from users, means for inputting the received detailed information into a generation AI model to generate renovation ideas, means for generating completed design images and videos using an image generation model based on the generated ideas, and means for returning the generated renovation ideas and design images and videos to the user via a smartphone application. This allows users to easily check specific renovation ideas and their visual images, and use them to plan for regional revitalization.

[0570] "Regional population data" refers to information about the population size, population density, age distribution, etc., within a specific region.

[0571] "Traffic data" refers to information regarding traffic volume, congestion, and the operating status of public transportation within a specific area.

[0572] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, entertainment facilities, and other facilities located within a specific area.

[0573] "People flow data" refers to information about the movement and dwell time of people within a specific area, as well as the number of visitors and their travel routes.

[0574] "Preprocessing" refers to the process of preparing collected data for training AI models by performing actions such as imputing missing values, removing outliers, and standardizing the data.

[0575] A "generative AI model" is an artificial intelligence model that learns from collected and pre-processed data to generate optimal utilization ideas for sparsely populated buildings.

[0576] "Detailed information on underutilized buildings from users" refers to specific information such as the location of the underutilized building, its current use, size, photos, and any wishes or conditions regarding renovation.

[0577] An "image generation model" is an artificial intelligence model that generates design images and videos of a renovated space based on ideas generated by a generative AI model.

[0578] A "smartphone application" is application software that runs on a smartphone and allows users to input detailed information about renovations and view renovation ideas, design images, and videos returned from a server.

[0579] "Promotional materials" refer to the generated renovation ideas and the design images and videos based on them, which will be used for promotional activities aimed at revitalizing the local community.

[0580] This invention is a system that utilizes generative AI models to renovate buildings in sparsely populated areas and support regional revitalization. The system is divided into three main components: a server, terminals, and users.

[0581] server

[0582] Data Acquisition and Preprocessing

[0583] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This data is preprocessed, including imputing missing values, removing outliers, and standardization, to prepare it as training data. Data processing libraries such as Python and Pandas are used for collection and preprocessing.

[0584] Training of Generative AI Models

[0585] The server trains a generative AI model using preprocessed data. Deep learning frameworks such as TensorFlow and PyTorch are used to train the AI ​​model. This AI model generates optimal utilization ideas for sparsely populated buildings. During the training process, the data is divided into training data and validation data, and the model's performance is evaluated.

[0586] Processing the request

[0587] The server receives detailed information about sparsely populated buildings sent by users via their devices. This includes the building's location, current usage, and any wishes or conditions for renovation. The server inputs the received information into an AI model to generate renovation ideas.

[0588] Idea generation and promotional material creation

[0589] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. Technologies such as Generative Opposite Networks (GANs) are used for image generation. The generated renovation ideas and design images and videos are returned to the user via a smartphone application.

[0590] terminal

[0591] User Interface

[0592] The terminal provides users with an interface for inputting detailed information about sparsely populated buildings. Users can use the terminal to input information such as the building's location, current photos, and renovation requests. This allows users to easily send the necessary information to the server.

[0593] Send Request

[0594] The terminal formats the information entered by the user into a data format and sends it to the server. The HTTPS protocol is used to send requests, ensuring secure and efficient data exchange.

[0595] Displaying Results

[0596] The server returns renovation ideas and promotional materials (design images and videos) to the user. This allows the user to create a concrete renovation plan.

[0597] User

[0598] Entering information

[0599] Users input detailed information about sparsely populated buildings they are interested in through the interface of a smartphone application. This includes, for example, the building's location, size, current use, photos, and their wishes and requirements for renovation.

[0600] Idea verification

[0601] Users can review renovation ideas and promotional materials displayed on their devices. Based on the suggested ideas, users can proceed with developing concrete project plans. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0602] Specific example

[0603] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information, current condition (with photos), and desired use (e.g., to use it as a community center) of the vacant school building through a smartphone application interface. The server receives this information and uses a generation AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated building. These results are returned to the user via the smartphone application, allowing the user to review them and develop a concrete renovation plan.

[0604] Example of a prompt

[0605] "This building is currently vacant. We would like to renovate it and turn it into a shop selling local specialty products. Please generate a renovation plan for the building, create an image rendering and a promotional video. We have attached some reference photos."

[0606] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[0607] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0608] Step 1:

[0609] Data collection (server)

[0610] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed using the Python requests library, and database management systems such as SQLite and PostgreSQL are used for queries from the databases. Inputs are external API endpoints and database queries, and outputs are the respective data.

[0611] Step 2:

[0612] Data preprocessing (server)

[0613] The collected data is preprocessed. This preprocessing includes imputing missing values, removing outliers, and standardization. The program uses the Pandas library to perform data cleaning. Specifically, missing values ​​are imputed with the median, and outliers (e.g., data exceeding ±3σ) are removed. The input is the collected raw data, and the output is the preprocessed data.

[0614] Step 3:

[0615] Training of generative AI models (server)

[0616] The server trains a generative AI model using preprocessed data. It builds and trains the model using deep learning frameworks such as TensorFlow and PyTorch. The data is divided into training and validation data, and the model's performance is evaluated throughout the training process. The input is preprocessed data, and the output is the trained generative AI model.

[0617] Step 4:

[0618] Receiving user input information (terminal)

[0619] The terminal provides a user interface, allowing users to input detailed information about sparsely populated buildings. This information includes the building's location, current use, size, photos, and any desired renovations or requirements. The input is the building information entered by the user, and the output is the data organized for transmission to the server.

[0620] Step 5:

[0621] Sending a request (from the terminal)

[0622] The terminal organizes the information entered by the user and generates a request to send to the server. The request contains all the collected information in JSON format. The input is the building information entered by the user, and the output is the request data sent to the server.

[0623] Step 6:

[0624] Renovation idea generation (server)

[0625] The server inputs detailed information received from the user into a generating AI model to generate renovation ideas. During this process, prompt statements are created and input into the model. The input is the user's building information, and the output is the generated renovation ideas.

[0626] Step 7:

[0627] Design image and video generation (server)

[0628] Based on the generated renovation ideas, the server uses an image generation model to produce images and videos of the completed design. This process employs generative opposite-party network (GAN) technology. The input is the renovation idea, and the output is the generated images and videos.

[0629] Step 8:

[0630] Return of results (server)

[0631] The generated renovation ideas, design images, and videos are returned to the user via a smartphone application. The input is design images and videos, and the output is the result displayed on the user's device.

[0632] Step 9:

[0633] Checking the results (user)

[0634] Users view renovation ideas and promotional materials (images and videos) displayed on their devices. Input is information returned from the server, and output is the visual content the user reviews. This allows for the creation of concrete renovation plans.

[0635] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0636] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[0637] server

[0638] Data Acquisition and Preprocessing

[0639] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. The collected data is preprocessed, including imputation of missing values, removal of outliers, and standardization, to prepare it for training AI models.

[0640] Training of Generative AI Models

[0641] The server uses pre-processed data to train a generative AI model that generates optimal utilization ideas for sparsely populated buildings. Using a deep learning framework (e.g., TensorFlow or PyTorch), the model is trained on training data and validation data, and its performance is evaluated.

[0642] Processing the request

[0643] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. The received information includes the building's location, current usage, and the user's wishes and requirements. The server inputs this detailed information into a generating AI model to generate renovation ideas.

[0644] Emotional engine integration

[0645] The emotion engine analyzes the user's emotional data and works in conjunction with a generative AI model to optimize renovation idea recommendations. Based on the information entered by the user and the emotional data analyzed by the emotion engine, more personalized renovation ideas are generated.

[0646] Idea generation and promotional material creation

[0647] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated ideas and promotional materials are then returned to the terminal.

[0648] terminal

[0649] User Interface

[0650] The device provides users with an interface for inputting detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, renovation wishes and conditions, and even their own feelings.

[0651] Send Request

[0652] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[0653] Displaying Results

[0654] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[0655] User

[0656] Entering information

[0657] Through the terminal interface, users can input detailed information and emotional data about sparsely populated buildings that interest them. For example, they can input the building's location, size, and current use, as well as their wishes and conditions regarding renovations, and their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[0658] Idea verification

[0659] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0660] Specific example

[0661] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. Through the terminal interface, the user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project).

[0662] The server receives this information and uses a generative AI model to generate optimal renovation ideas. The emotion engine analyzes the user's emotional data and reflects the results in the generative AI model, providing personalized ideas that take the user's emotions into consideration. Based on the generated ideas, the image generation model creates design images and videos of the renovated space. These results are returned to the device, allowing the user to review them and develop a concrete plan.

[0663] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[0664] The following describes the processing flow.

[0665] Step 1:

[0666] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This allows for accurate and up-to-date information about sparsely populated areas.

[0667] Step 2:

[0668] The server preprocesses the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. In this way, it prepares the data in a format suitable for training.

[0669] Step 3:

[0670] The server uses pre-processed data to train a generative AI model. A deep learning framework (such as TensorFlow or PyTorch) is used for training, dividing the data into training and validation sets to train the model, and then its performance is evaluated.

[0671] Step 4:

[0672] Users input detailed information about sparsely populated buildings through the terminal's user interface. This information includes location data, current photos, renovation wishes and requirements, and even emotional data (e.g., excitement, anticipation, anxiety).

[0673] Step 5:

[0674] The terminal formats the information entered by the user and sends it to the server. The terminal performs formatting and error checking to ensure that the data is transmitted accurately.

[0675] Step 6:

[0676] The server receives detailed information and sentiment data about sparsely populated buildings sent from the terminal. The received information includes the building's location and details about its usage.

[0677] Step 7:

[0678] The server inputs the received detailed information and emotional data into the generating AI model and emotion engine to generate optimal renovation ideas for sparsely populated buildings. In this process, the emotion engine analyzes the user's emotions and reflects the results in the generating AI model.

[0679] Step 8:

[0680] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. The generated designs are mindful of the user's emotions.

[0681] Step 9:

[0682] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is formatted into a user-friendly format.

[0683] Step 10:

[0684] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[0685] Step 11:

[0686] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[0687] (Example 2)

[0688] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0689] In the renovation of buildings in sparsely populated areas, conventional methods failed to effectively utilize region-specific data, and it was difficult to provide personalized proposals that reflected user sentiment. This made it difficult to generate appropriate utilization ideas and to realize concrete plans that would contribute to regional revitalization. In particular, proposals that did not consider user sentiment were problematic because they made it difficult to motivate actual initiatives and gain the support of stakeholders.

[0690] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0691] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for analyzing the user's emotional data; and means for reflecting the analyzed emotional data in the generative AI model. This makes it possible to generate personalized renovation ideas that reflect region-specific data and the user's emotions.

[0692] "Regional population data" refers to information about the distribution and demographic trends of the population in a specific region.

[0693] "Traffic data" refers to information about traffic flow, congestion levels, and traffic volume in a specific area.

[0694] "Neighborhood facility data" refers to information about the location and characteristics of public facilities, commercial facilities, educational facilities, etc., that exist within a specific area.

[0695] "Human flow data" refers to information about the movement patterns and flow of people in a specific area.

[0696] "Preprocessing" refers to processes such as imputing missing values, removing outliers, and normalizing data in order to prepare it for analysis and model training.

[0697] A "generative AI model" is a model that uses artificial intelligence technology to generate a desired output from specific input data.

[0698] A "renovation idea" is a specific plan or proposal for remodeling or renovating an existing building.

[0699] "Training a generative AI model" refers to the training process of adjusting the parameters of a generative AI model using appropriate input data to obtain highly accurate output.

[0700] "User emotional data" refers to information about the emotions a user is experiencing, such as data on various emotions like anticipation, anxiety, and excitement.

[0701] An "emotion engine" is a system that analyzes the emotions of a user based on the data they input and outputs them in a specific format.

[0702] An "image generation model" is an artificial intelligence model that generates visual materials such as images and videos based on specific input data.

[0703] A "user interface" refers to the means of operation, such as screens and forms, that a user uses to input information into a system.

[0704] "Promotional materials" refer to visual and text content used to advertise a project or proposal, such as generated renovation ideas and design images.

[0705] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user, each with its own role and specific processing as follows.

[0706] Server Processing

[0707] Data Acquisition and Preprocessing

[0708] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from statistical agency APIs, and traffic data is obtained from traffic information services. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. Methods such as imputing missing values ​​with the mean or mode are used. Outlier removal is performed by deleting data that is statistically determined to be outliers. Finally, the data is standardized to shape it into a form suitable for model training.

[0709] Training of Generative AI Models

[0710] The server uses pre-processed data to train a generative AI model to generate optimal utilization ideas for sparsely populated buildings. The deep learning frameworks used include TensorFlow and PyTorch. During training, the dataset is divided into training and validation data, and the model's performance is evaluated and optimized. For example, 80% of the data might be used as training data and 20% as validation data.

[0711] Processing the request

[0712] The server receives detailed information about sparsely populated buildings sent from users via their devices. This information includes the building's location, current usage, and the user's wishes and requirements. This information is then input into an AI model to generate renovation ideas. For example, if the target is a closed school in XX City, Nagano Prefecture, the server would provide information such as its location, current usage, and the user's wishes (e.g., wanting to use it as a community center).

[0713] Emotional engine integration

[0714] The server uses an emotion engine to analyze the user's emotional data. The emotion engine uses NLP (Neuro-Linguistic Programming) techniques to extract the user's emotions from text and reflects that data in the generating AI model. For example, if a user inputs "I have high hopes for this project, but I also have some anxieties," the emotions of "excitement" and "anxiety" will be analyzed.

[0715] Idea generation and promotional material creation

[0716] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the completed project. For example, it uses a GAN (Generative Opposite Network) to generate realistic design images. The generated ideas and design images are used as promotional materials and returned to the terminal.

[0717] Terminal processing

[0718] User Interface

[0719] The device provides users with an interface for entering detailed information about sparsely populated buildings. This might include forms or photo upload functions. Users use this interface to input location information, current photos, desired usage, and their feelings.

[0720] Send Request

[0721] The terminal formats the information entered by the user and sends it to the server. For example, it converts the data to JSON format, performs error checking, and then sends it.

[0722] Displaying Results

[0723] The server returns renovation ideas and promotional materials (design images and videos) to the user. The terminal provides a viewer to display the generated design images and videos, making it easy for the user to review them.

[0724] User processing

[0725] Entering information

[0726] Through the terminal interface, users input information about the building's location, size, current use, and their wishes, conditions, and emotions (excitement, anticipation, anxiety, etc.) regarding the renovation. For example, they might input their wish to "use a closed school in XX City, Nagano Prefecture as a community center," along with photos of the current state and their "expectations and anxieties about the project."

[0727] Idea verification

[0728] Review the proposed renovation ideas and promotional materials. View the generated design images and videos to develop a concrete renovation plan. For example, determine the project's progress based on the design images and videos displayed on the device.

[0729] Example of a prompt

[0730] "We want to use a closed school located in a sparsely populated area as a community center. Please generate renovation ideas based on the following information:

[0731] Building location: Nagano Prefecture, XX City, △△ Town

[0732] Current photos: xxx.jpg, yyy.jpg

[0733] Current use: A closed elementary school

[0734] User emotions: Expectation (want to contribute to regional revitalization), anxiety (worried about whether it will succeed)

[0735] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. Furthermore, the introduction of an emotion engine enables personalized suggestions that take into account the user's emotions.

[0736] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0737] Step 1: Data Collection

[0738] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, it uses APIs from statistical agencies and traffic information services.

[0739] Input: Raw data from external APIs or databases.

[0740] Output: Collected raw data.

[0741] Step 2: Data Preprocessing

[0742] The server performs missing value imputation, outlier removal, and standardization on the collected data. For example, it imputates missing values ​​with the mean, statistically removes outliers, and normalizes numerical data from 0 to 1.

[0743] Input: Collected raw data.

[0744] Output: Pre-processed data.

[0745] Step 3: Training the Generative AI Model

[0746] The server uses pre-processed data to train a generative AI model. For example, it can use TensorFlow or PyTorch to prepare training and validation data and then train the model.

[0747] Input: Pre-processed data.

[0748] Output: A trained generative AI model.

[0749] Step 4: Receiving User Information

[0750] The server receives detailed information about sparsely populated buildings sent from users via their terminals. This includes, for example, the building's location, current usage, and the user's preferences and requirements.

[0751] Input: Building information submitted by the user.

[0752] Output: Detailed information stored on the server.

[0753] Step 5: Generating renovation ideas

[0754] The server inputs the received detailed information into a generating AI model to produce renovation ideas. For example, it proposes the optimal renovation plan based on the building's location and intended use.

[0755] Input: Detailed information and a trained generative AI model.

[0756] Output: Renovation ideas.

[0757] Step 6: Analyzing emotional data

[0758] The server analyzes emotional data from users using an emotion engine. For example, it extracts emotions from text using NLP (Neuro-Linguistic Programming) techniques.

[0759] Input: User sentiment data.

[0760] Output: Analyzed sentiment data.

[0761] Step 7: Personalized Idea Generation

[0762] The server incorporates the analyzed emotional data into a generating AI model to create personalized ideas. It provides renovation ideas that reflect the user's emotions.

[0763] Input: Analyzed sentiment data and detailed information.

[0764] Output: Personalized renovation ideas.

[0765] Step 8: Create promotional materials

[0766] The server uses image generation models to create design images and videos based on the generated renovation ideas. For example, it uses GANs to generate realistic images of the finished product.

[0767] Input: Generated renovation ideas.

[0768] Output: Promotional materials (design images, videos).

[0769] Step 9: Return of Results

[0770] The server returns the generated renovation ideas and promotional materials to the terminal and provides them to the user.

[0771] Input: Promotional material.

[0772] Output: Data returned to the terminal.

[0773] Step 10: User Input

[0774] Users input detailed information such as the building's location, photos, desired usage, and emotions through the device's interface.

[0775] Input: Various data entered by the user.

[0776] Output: Detailed data sent to the terminal.

[0777] Step 11: Submit Request

[0778] The terminal formats the information entered by the user and sends it to the server. For example, it converts the information to JSON format, performs error checking, and then sends it.

[0779] Input: Detailed information entered by the user.

[0780] Output: Request data sent to the server.

[0781] Step 12: Displaying the results

[0782] The terminal displays renovation ideas and promotional materials returned from the server to the user. For example, it provides a viewer that allows users to view generated design images and videos.

[0783] Input: Data returned from the server.

[0784] Output: Data displayed to the user.

[0785] (Application Example 2)

[0786] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0787] It is crucial to effectively renovate buildings in sparsely populated areas and promote regional revitalization. However, what is needed is not just the generation of renovation ideas, but also personalized proposals that take into account the user's feelings. Furthermore, there is a lack of visual means to convey how the generated ideas can be specifically useful. To solve these problems, a system is needed that proposes the optimal utilization methods for sparsely populated buildings, as well as providing optimal renovation ideas and promotional materials based on the user's feelings.

[0788] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0789] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; and means including an emotion engine that analyzes emotion data and works in conjunction with the generative AI model to optimize renovation ideas based on the user's emotions. This makes it possible to provide personalized renovation ideas and concrete visual images that meet the user's wishes and emotions.

[0790] "Regional population data" refers to statistical information such as the number of people residing in a specific area, the number of households, age groups, and gender.

[0791] "Traffic data" refers to information such as traffic volume, congestion levels, and the operating status of public transportation in a specific area.

[0792] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, public facilities, and other similar facilities located in a specific area.

[0793] "People flow data" refers to information such as people's movement patterns, flow conditions, and ability to attract customers in a specific area.

[0794] "Preprocessing" refers to the data organization and editing work required to apply large amounts of data to machine learning models.

[0795] A "generative AI model" refers to an artificial intelligence model that uses machine learning to generate optimal renovation ideas from data.

[0796] "Detailed information" refers to information provided by users, such as the location, size, current usage, and desired conditions or requirements for renovations of sparsely populated buildings.

[0797] "Emotional data" refers to information about a user's psychological state and emotions (e.g., enjoyment, anticipation, anxiety) obtained from user input, facial expressions, voice, and behavior.

[0798] An "emotion engine" refers to a system that analyzes user emotional data and incorporates the results into the generated AI model.

[0799] An "image generation model" refers to a model that uses AI to generate actual images from text and other data.

[0800] "Renovation ideas" refer to specific proposals and design plans for how to remodel and reuse sparsely populated buildings.

[0801] "Personalized suggestions" refer to providing individual renovation ideas optimized based on each user's wishes and feelings.

[0802] "Promotional materials" refer to advertising and public relations content created using generated renovation ideas and design images.

[0803] This invention is a system that generates and provides renovation ideas for buildings in sparsely populated areas, thereby supporting regional revitalization. This system consists of the following three main components:

[0804] server

[0805] The server first collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from the census database, traffic data from the traffic management system, nearby facilities data from the Geographic Information System (GIS), and pedestrian flow data from the mobile data analytics service.

[0806] The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardization. For data preprocessing, the Python pandas library is used, for example.

[0807] Using pre-processed data, the server trains a generative AI model using a deep learning framework (TensorFlow or PyTorch). This model is used to generate optimal utilization ideas for sparsely populated buildings. This allows the model to learn diverse data patterns and produce highly accurate generation results.

[0808] When a user sends detailed information about a sparsely populated building from their device, the server receives this information and inputs it into the generated AI model. This detailed information includes the building's location, current usage, and the user's wishes and conditions regarding renovations.

[0809] Furthermore, the server uses an emotion engine to analyze the user's emotional data and integrates this result with a generating AI model to create more personalized renovation ideas. Based on the analyzed emotional data, the server makes suggestions that take into account the user's expectations and anxieties.

[0810] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the finished product. This allows users to have a concrete visual image of the project. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream.

[0811] terminal

[0812] The terminal provides users with a user interface for entering detailed information about sparsely populated buildings. Through this interface, users can input the building's location, current photos, and their wishes and requirements for renovation. Furthermore, they can also input their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[0813] The information entered by the user is formatted into the appropriate format by the terminal and sent to the server. The terminal also performs error checking on the information to ensure that the data is transmitted accurately.

[0814] Renovation ideas and promotional materials (images and videos) returned from the server are displayed on the device. This allows users to create concrete renovation plans based on these materials.

[0815] User

[0816] Users input detailed information and sentimental data about sparsely populated buildings they are interested in via their devices. For example, they input the building's location, size, and current use, as well as their wishes, conditions, and feelings regarding renovations (e.g., excitement, anticipation, anxiety).

[0817] Users can view renovation ideas and promotional materials displayed on their devices and use them to develop concrete project plans. The generated materials can also be used on social media and in presentations, serving as valuable tools to attract the attention of local communities and stakeholders.

[0818] Specific example

[0819] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. The user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project) from their device.

[0820] The server receives this information and generates renovation ideas using a generative AI model. The emotion engine analyzes the user's emotional data and incorporates the results into the generative AI model, providing personalized ideas that take the user's emotions into consideration.

[0821] Example of a prompt

[0822] User input:

[0823] Building details: Abandoned school in Nagano Prefecture, area 1200m² 2 Currently used as a warehouse.

[0824] Renovation request: I would like to use it as a community center.

[0825] Emotions: I'm very excited, but also a little anxious.

[0826] Expected output:

[0827] Generated from AI model: "By renovating the abandoned school into a community center, local interaction can be deepened and it can be used as a new tourist attraction. In particular, it is important to incorporate a design that prioritizes comfort. Since users have given positive feedback, let's give maximum consideration to comfort and functionality."

[0828] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[0829] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0830] Step 1:

[0831] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. For example, population data is obtained from a census database, and traffic data is obtained from a traffic management system. The input is raw data obtained from APIs and databases, and the output is preprocessed, formatted data.

[0832] Step 2:

[0833] The server uses preprocessed data to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). This allows the server to build a generative AI model for generating optimal utilization ideas for sparsely populated buildings. The input is preprocessed data, and the output is the trained generative AI model. Specifically, the model is trained using training data, and its performance is evaluated.

[0834] Step 3:

[0835] Users input detailed information about sparsely populated buildings and their own feelings using a user interface on their device. This might include the building's location, current photos, renovation hopes and conditions, and emotions (e.g., excitement, anticipation, anxiety). The input is information provided by the user, and the output is formatted detailed data.

[0836] Step 4:

[0837] The terminal formats the user-entered details into the appropriate format and sends them to the server. It also performs error checking to ensure the data is sent accurately to the server. The input is the information entered by the user, and the output is the formatted data after error checking.

[0838] Step 5:

[0839] The server receives detailed information sent by the user and inputs this information into the generating AI model. Specifically, it supplies the AI ​​model with information such as the building's location, usage, and the user's wishes and conditions. The input is the detailed information received from the user, and the output is the generated renovation ideas.

[0840] Step 6:

[0841] The server analyzes emotional data and works with a generative AI model to optimize renovation ideas based on the user's emotions. The emotion engine analyzes user input, facial expressions, voice, etc., and provides emotion-based data. The input is the user's emotional data, and the output is an optimized renovation idea based on those emotions.

[0842] Step 7:

[0843] The server uses an image generation model to generate images and videos of the completed design based on the generated renovation ideas. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream. The input is the generated renovation idea, and the output is the images and videos of the completed design.

[0844] Step 8:

[0845] The server returns the generated renovation ideas and design images to the terminal. This allows the user to have a concrete visual image. The input is the generated renovation ideas and design images, and the output is the data returned to the user.

[0846] Step 9:

[0847] The terminal displays renovation ideas and promotional materials (images and videos) returned from the server to the user. This allows the user to create a concrete renovation plan based on these materials. The input is data returned from the server, and the output is the renovation ideas and design images displayed to the user.

[0848] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0849] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0850] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0851] [Third Embodiment]

[0852] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0853] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0854] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0855] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0856] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0857] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0858] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0859] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0860] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0861] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0862] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0863] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0864] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[0865] server

[0866] Data Acquisition and Preprocessing

[0867] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. It then performs preprocessing on the collected data, such as imputing missing values, removing outliers, and standardizing the data. This prepares the data for training.

[0868] Training of Generative AI Models

[0869] The server trains a generative AI model using preprocessed data. This AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used for training, and the data is divided into training and validation data for model training. Model evaluation is also performed at this stage to verify its performance.

[0870] Processing the request

[0871] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. This information includes the building's location and current usage. The server inputs this information into an AI model to generate ideas for its utilization.

[0872] Idea generation and promotional material creation

[0873] Based on the generated utilization ideas, the server uses an image generation model to create images and videos that visually represent the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated utilization ideas and promotional materials are then returned to the terminal.

[0874] terminal

[0875] User Interface

[0876] The terminal provides users with an interface for entering detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, and their wishes and conditions regarding renovations.

[0877] Send Request

[0878] To send the information entered by the user to the server, the terminal formats the data and sends the request.

[0879] Displaying Results

[0880] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[0881] User

[0882] Entering information

[0883] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This includes, for example, the building's location, size, and current use. Users can also enter their wishes and conditions regarding renovations.

[0884] Idea verification

[0885] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[0886] Specific example

[0887] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information of the vacant school building, its current condition (with photos), and their desired use (for example, to use it as a community center) through the terminal interface.

[0888] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated space. These results are returned to the terminal, allowing the user to review them and develop a concrete renovation plan.

[0889] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[0890] The following describes the processing flow.

[0891] Step 1:

[0892] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed regularly with the aim of obtaining accurate and up-to-date information.

[0893] Step 2:

[0894] The server performs preprocessing on the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. This prepares a dataset suitable for subsequent AI model training.

[0895] Step 3:

[0896] The server trains a generative AI model using preprocessed data. It splits the data into training data and validation data, trains the model using a deep learning framework (e.g., TensorFlow or PyTorch), and evaluates the model's performance.

[0897] Step 4:

[0898] Users input detailed information about sparsely populated buildings (location, current photos, renovation requests and conditions, etc.) through the terminal's user interface. Once the input is complete, they submit a request for renovation idea generation.

[0899] Step 5:

[0900] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[0901] Step 6:

[0902] The server receives detailed information about sparsely populated buildings sent from the terminal. The received information includes the building's location and details on how to use it.

[0903] Step 7:

[0904] The server inputs the received information into a generating AI model, which then generates optimal renovation ideas for sparsely populated buildings. This AI model proposes the best plan based on pre-trained data.

[0905] Step 8:

[0906] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. This provides users and stakeholders with concrete visual images.

[0907] Step 9:

[0908] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is then formatted to be useful to the user.

[0909] Step 10:

[0910] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[0911] Step 11:

[0912] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[0913] (Example 1)

[0914] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0915] There is a problem in sparsely populated areas where the utilization of buildings is not progressing, leading to a stagnation of regional revitalization. Furthermore, current renovation plans are not concrete, making it difficult to attract the interest of stakeholders and reach a consensus for implementation. In this situation, there is a need for a system that can quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images to support the realization of renovation plans.

[0916] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0917] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for generating post-renovation design images and videos as promotional materials; means for providing users with concrete visual images to help them plan renovations; and means for providing the generated products as materials for social media and presentations. This makes it possible to quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images.

[0918] "Regional population data" refers to information about the number of people living in a specific area, population density, age distribution, and so on.

[0919] "Traffic data" refers to information regarding traffic volume, the number of traffic accidents, and the types of vehicles traveling in a specific area.

[0920] "Nearby facilities data" refers to information about the location, number, and types of schools, hospitals, supermarkets, public facilities, etc., that exist in a specific area.

[0921] "Human flow data" refers to information that shows the movement patterns and flow of people in a specific area.

[0922] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze, such as imputing missing values, removing outliers, and standardizing the data.

[0923] A "generative AI model" refers to an artificial intelligence model that uses machine learning and deep learning techniques to generate useful information and ideas from new data.

[0924] "Detailed information on sparsely populated buildings from users" refers to information including location information, current usage status, and desired renovations for buildings in sparsely populated areas where renovation is being considered.

[0925] "Renovation ideas" refer to ideas that use generative AI models to suggest optimal ways to utilize or renovate sparsely populated buildings.

[0926] An "image generation model" refers to a model that generates visual design images and videos based on ideas proposed by a generative AI model.

[0927] "Promotional materials" refer to images and videos that visually demonstrate the design and usage of a renovated building, and are intended to attract the attention of project stakeholders and the general public.

[0928] "Visual images" refer to images or videos that visually embody the generated renovation ideas.

[0929] "Materials for social media and presentations" refers to digital files such as images and videos used in social media posts and presentations to promote renovation ideas.

[0930] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each are described below.

[0931] server

[0932] Data Acquisition and Preprocessing

[0933] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data. Specifically, it obtains this data from external APIs (e.g., location APIs and traffic APIs) and databases (e.g., local databases). The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardizing the data. Software tools such as the Python Pandas library are used for preprocessing.

[0934] Training of Generative AI Models

[0935] The server trains a generative AI model using pre-processed data. The generative AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used to train the model, and the data is split into training and validation sets for training.

[0936] Receiving user requests

[0937] The server receives detailed information about sparsely populated buildings entered by the user from their device. This information includes the building's location, current usage, and renovation requests and conditions. This data is sent to the server via an HTTP request.

[0938] Idea generation using AI models

[0939] The server inputs the received detailed information into a generating AI model to generate optimal renovation ideas. Based on the input data, the generating AI model proposes specific ways to utilize the property.

[0940] Creating promotional materials

[0941] The server uses an image generation model to create design images and videos of the completed project based on the generated renovation ideas. This promotional material visually showcases the generated ideas and utilizes image generation technologies such as DeepArt and MidJourney.

[0942] Return of renovation ideas

[0943] The server returns the generated renovation ideas and design images to the user, allowing them to develop a concrete renovation plan.

[0944] terminal

[0945] User Interface

[0946] The terminal provides an interface that allows users to input detailed information about sparsely populated buildings. Through this interface, users can enter details such as the building's location, current photos, and desired renovations.

[0947] Send Request

[0948] The terminal sends the user's entered details to the server. The transmission occurs via an HTTP request, and the data is formatted into the appropriate format.

[0949] Displaying Results

[0950] The terminal displays renovation ideas and design images returned from the server to the user, providing information to help them move forward with concrete project planning.

[0951] User

[0952] Entering information

[0953] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This information includes the building's location, current photos, and their wishes and requirements regarding renovations.

[0954] Idea verification

[0955] Users will view renovation ideas and design images displayed on their devices and use them to create concrete renovation plans. They will also use the generated promotional materials on social media and in presentations to attract the attention of stakeholders.

[0956] Examples of specific cases and prompt statements

[0957] For example, if the user were to use this system to generate renovation ideas for a vacant school building located in a sparsely populated area, they would enter the following prompt:

[0958] "The location of the vacant school building is [latitude and longitude]. Its current use is [description]. The desired use after renovation is a community center. Please generate the most suitable utilization ideas."

[0959] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Then, an image generation model creates design images and videos of the renovated space, and these results are returned to the user's device. The user can then review them and proceed with a concrete renovation plan.

[0960] This will enable the effective support of regional revitalization through a system that generates renovation ideas for sparsely populated buildings using a generation AI model.

[0961] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0962] Step 1: Data Collection

[0963] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. Specifically, the server uses location APIs and traffic APIs to obtain data for a specified area. For example, it obtains local population data from a census API and traffic data from a traffic API. Input requires an API request, and output is data returned in JSON format.

[0964] Step 2: Data Preprocessing

[0965] The server preprocesses the collected data. Specifically, it uses the Python Pandas library to impute missing values, remove outliers, and standardize the data. The data collected in step 1 is used as input, and the output is clean, preprocessed data. This is to prepare the data for training the generative AI model.

[0966] Step 3: Training the Generative AI Model

[0967] The server uses preprocessed data to train a generative AI model. Specifically, it trains a deep learning model using TensorFlow or PyTorch. Preprocessed data is used as input, and the trained generative AI model is obtained as output. Furthermore, the model is evaluated at this stage to confirm its performance.

[0968] Step 4: Receiving User Requests

[0969] The terminal sends detailed information about sparsely populated buildings entered by the user to the server. Specifically, it sends information obtained through the user interface as an HTTP request. The input is information entered by the user (location information, current photos, renovation requests), and the output is received by the server.

[0970] Step 5: Idea generation using AI models

[0971] The server inputs the received user details into a generating AI model to generate optimal renovation ideas. Specifically, the generating AI model proposes utilization methods based on the input data. The input uses detailed information received from the user, and the output is the generated renovation ideas.

[0972] Step 6: Create promotional materials

[0973] The server uses an image generation model to create design images and videos based on the generated renovation ideas. Specifically, it utilizes tools such as DeepArt and MidJourney to generate visuals of the renovated area. The generated renovation ideas are used as input, and promotional materials in image and video formats are obtained as output.

[0974] Step 7: Return of renovation ideas

[0975] The server returns the generated renovation ideas and promotional materials to the terminal. Specifically, it formats the generated content and sends it to the terminal as an HTTP response. The generated content (ideas and materials) is used as input, and the output is sent back to the terminal.

[0976] Step 8: Displaying the results

[0977] The terminal displays renovation ideas and promotional materials returned from the server to the user. Specifically, it displays the returned data on the user interface and formats it so that the user can review it. The input is the data returned from the server, and the output is displayed in a format that the user can view. The user then uses this to create a concrete renovation plan.

[0978] (Application Example 1)

[0979] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0980] Renovating buildings in sparsely populated areas is difficult due to population decline and decreased demand. Furthermore, the process of proposing concrete renovation ideas and designs is cumbersome, highlighting the need for effective methods of regional revitalization. Therefore, a system is needed to quickly and efficiently generate and provide renovation ideas and concrete designs for physical stores in sparsely populated areas.

[0981] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0982] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; and means for learning a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data. This makes it possible to quickly and efficiently generate renovation ideas for physical stores located in sparsely populated areas and provide users with design images and videos based on those ideas.

[0983] The server also includes means for receiving detailed information about sparsely populated buildings from users, means for inputting the received detailed information into a generation AI model to generate renovation ideas, means for generating completed design images and videos using an image generation model based on the generated ideas, and means for returning the generated renovation ideas and design images and videos to the user via a smartphone application. This allows users to easily check specific renovation ideas and their visual images, and use them to plan for regional revitalization.

[0984] "Regional population data" refers to information about the population size, population density, age distribution, etc., within a specific region.

[0985] "Traffic data" refers to information regarding traffic volume, congestion, and the operating status of public transportation within a specific area.

[0986] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, entertainment facilities, and other facilities located within a specific area.

[0987] "People flow data" refers to information about the movement and dwell time of people within a specific area, as well as the number of visitors and their travel routes.

[0988] "Preprocessing" refers to the process of preparing collected data for training AI models by performing actions such as imputing missing values, removing outliers, and standardizing the data.

[0989] A "generative AI model" is an artificial intelligence model that learns from collected and pre-processed data to generate optimal utilization ideas for sparsely populated buildings.

[0990] "Detailed information on underutilized buildings from users" refers to specific information such as the location of the underutilized building, its current use, size, photos, and any wishes or conditions regarding renovation.

[0991] An "image generation model" is an artificial intelligence model that generates design images and videos of a renovated space based on ideas generated by a generative AI model.

[0992] A "smartphone application" is application software that runs on a smartphone and allows users to input detailed information about renovations and view renovation ideas, design images, and videos returned from a server.

[0993] "Promotional materials" refer to the generated renovation ideas and the design images and videos based on them, which will be used for promotional activities aimed at revitalizing the local community.

[0994] This invention is a system that utilizes generative AI models to renovate buildings in sparsely populated areas and support regional revitalization. The system is divided into three main components: a server, terminals, and users.

[0995] server

[0996] Data Acquisition and Preprocessing

[0997] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This data is preprocessed, including imputing missing values, removing outliers, and standardization, to prepare it as training data. Data processing libraries such as Python and Pandas are used for collection and preprocessing.

[0998] Training of Generative AI Models

[0999] The server trains a generative AI model using preprocessed data. Deep learning frameworks such as TensorFlow and PyTorch are used to train the AI ​​model. This AI model generates optimal utilization ideas for sparsely populated buildings. During the training process, the data is divided into training data and validation data, and the model's performance is evaluated.

[1000] Processing the request

[1001] The server receives detailed information about sparsely populated buildings sent by users via their devices. This includes the building's location, current usage, and any wishes or conditions for renovation. The server inputs the received information into an AI model to generate renovation ideas.

[1002] Idea generation and promotional material creation

[1003] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. Technologies such as Generative Opposite Networks (GANs) are used for image generation. The generated renovation ideas and design images and videos are returned to the user via a smartphone application.

[1004] terminal

[1005] User Interface

[1006] The terminal provides users with an interface for inputting detailed information about sparsely populated buildings. Users can use the terminal to input information such as the building's location, current photos, and renovation requests. This allows users to easily send the necessary information to the server.

[1007] Send Request

[1008] The terminal formats the information entered by the user into a data format and sends it to the server. The HTTPS protocol is used to send requests, ensuring secure and efficient data exchange.

[1009] Displaying Results

[1010] The server returns renovation ideas and promotional materials (design images and videos) to the user. This allows the user to create a concrete renovation plan.

[1011] User

[1012] Entering information

[1013] Users input detailed information about sparsely populated buildings they are interested in through the interface of a smartphone application. This includes, for example, the building's location, size, current use, photos, and their wishes and requirements for renovation.

[1014] Idea verification

[1015] Users can review renovation ideas and promotional materials displayed on their devices. Based on the suggested ideas, users can proceed with developing concrete project plans. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[1016] Specific example

[1017] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information, current condition (with photos), and desired use (e.g., to use it as a community center) of the vacant school building through a smartphone application interface. The server receives this information and uses a generation AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated building. These results are returned to the user via the smartphone application, allowing the user to review them and develop a concrete renovation plan.

[1018] Example of a prompt

[1019] "This building is currently vacant. We would like to renovate it and turn it into a shop selling local specialty products. Please generate a renovation plan for the building, create an image rendering and a promotional video. We have attached some reference photos."

[1020] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[1021] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1022] Step 1:

[1023] Data collection (server)

[1024] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed using the Python requests library, and database management systems such as SQLite and PostgreSQL are used for queries from the databases. Inputs are external API endpoints and database queries, and outputs are the respective data.

[1025] Step 2:

[1026] Data preprocessing (server)

[1027] The collected data is preprocessed. This preprocessing includes imputing missing values, removing outliers, and standardization. The program uses the Pandas library to perform data cleaning. Specifically, missing values ​​are imputed with the median, and outliers (e.g., data exceeding ±3σ) are removed. The input is the collected raw data, and the output is the preprocessed data.

[1028] Step 3:

[1029] Training of generative AI models (server)

[1030] The server trains a generative AI model using preprocessed data. It builds and trains the model using deep learning frameworks such as TensorFlow and PyTorch. The data is divided into training and validation data, and the model's performance is evaluated throughout the training process. The input is preprocessed data, and the output is the trained generative AI model.

[1031] Step 4:

[1032] Receiving user input information (terminal)

[1033] The terminal provides a user interface, allowing users to input detailed information about sparsely populated buildings. This information includes the building's location, current use, size, photos, and any desired renovations or requirements. The input is the building information entered by the user, and the output is the data organized for transmission to the server.

[1034] Step 5:

[1035] Sending a request (from the terminal)

[1036] The terminal organizes the information entered by the user and generates a request to send to the server. The request contains all the collected information in JSON format. The input is the building information entered by the user, and the output is the request data sent to the server.

[1037] Step 6:

[1038] Renovation idea generation (server)

[1039] The server inputs detailed information received from the user into a generating AI model to generate renovation ideas. During this process, prompt statements are created and input into the model. The input is the user's building information, and the output is the generated renovation ideas.

[1040] Step 7:

[1041] Design image and video generation (server)

[1042] Based on the generated renovation ideas, the server uses an image generation model to produce images and videos of the completed design. This process employs generative opposite-party network (GAN) technology. The input is the renovation idea, and the output is the generated images and videos.

[1043] Step 8:

[1044] Return of results (server)

[1045] The generated renovation ideas, design images, and videos are returned to the user via a smartphone application. The input is design images and videos, and the output is the result displayed on the user's device.

[1046] Step 9:

[1047] Checking the results (user)

[1048] Users view renovation ideas and promotional materials (images and videos) displayed on their devices. Input is information returned from the server, and output is the visual content the user reviews. This allows for the creation of concrete renovation plans.

[1049] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1050] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[1051] server

[1052] Data Acquisition and Preprocessing

[1053] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. The collected data is preprocessed, including imputation of missing values, removal of outliers, and standardization, to prepare it for training AI models.

[1054] Training of Generative AI Models

[1055] The server uses pre-processed data to train a generative AI model that generates optimal utilization ideas for sparsely populated buildings. Using a deep learning framework (e.g., TensorFlow or PyTorch), the model is trained on training data and validation data, and its performance is evaluated.

[1056] Processing the request

[1057] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. The received information includes the building's location, current usage, and the user's wishes and requirements. The server inputs this detailed information into a generating AI model to generate renovation ideas.

[1058] Emotional engine integration

[1059] The emotion engine analyzes the user's emotional data and works in conjunction with a generative AI model to optimize renovation idea recommendations. Based on the information entered by the user and the emotional data analyzed by the emotion engine, more personalized renovation ideas are generated.

[1060] Idea generation and promotional material creation

[1061] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated ideas and promotional materials are then returned to the terminal.

[1062] terminal

[1063] User Interface

[1064] The device provides users with an interface for inputting detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, renovation wishes and conditions, and even their own feelings.

[1065] Send Request

[1066] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[1067] Displaying Results

[1068] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[1069] User

[1070] Entering information

[1071] Through the terminal interface, users can input detailed information and emotional data about sparsely populated buildings that interest them. For example, they can input the building's location, size, and current use, as well as their wishes and conditions regarding renovations, and their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[1072] Idea verification

[1073] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[1074] Specific example

[1075] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. Through the terminal interface, the user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project).

[1076] The server receives this information and uses a generative AI model to generate optimal renovation ideas. The emotion engine analyzes the user's emotional data and reflects the results in the generative AI model, providing personalized ideas that take the user's emotions into consideration. Based on the generated ideas, the image generation model creates design images and videos of the renovated space. These results are returned to the device, allowing the user to review them and develop a concrete plan.

[1077] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[1078] The following describes the processing flow.

[1079] Step 1:

[1080] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This allows for accurate and up-to-date information about sparsely populated areas.

[1081] Step 2:

[1082] The server preprocesses the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. In this way, it prepares the data in a format suitable for training.

[1083] Step 3:

[1084] The server uses pre-processed data to train a generative AI model. A deep learning framework (such as TensorFlow or PyTorch) is used for training, dividing the data into training and validation sets to train the model, and then its performance is evaluated.

[1085] Step 4:

[1086] Users input detailed information about sparsely populated buildings through the terminal's user interface. This information includes location data, current photos, renovation wishes and requirements, and even emotional data (e.g., excitement, anticipation, anxiety).

[1087] Step 5:

[1088] The terminal formats the information entered by the user and sends it to the server. The terminal performs formatting and error checking to ensure that the data is transmitted accurately.

[1089] Step 6:

[1090] The server receives detailed information and sentiment data about sparsely populated buildings sent from the terminal. The received information includes the building's location and details about its usage.

[1091] Step 7:

[1092] The server inputs the received detailed information and emotional data into the generating AI model and emotion engine to generate optimal renovation ideas for sparsely populated buildings. In this process, the emotion engine analyzes the user's emotions and reflects the results in the generating AI model.

[1093] Step 8:

[1094] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. The generated designs are mindful of the user's emotions.

[1095] Step 9:

[1096] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is formatted into a user-friendly format.

[1097] Step 10:

[1098] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[1099] Step 11:

[1100] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[1101] (Example 2)

[1102] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1103] In the renovation of buildings in sparsely populated areas, conventional methods failed to effectively utilize region-specific data, and it was difficult to provide personalized proposals that reflected user sentiment. This made it difficult to generate appropriate utilization ideas and to realize concrete plans that would contribute to regional revitalization. In particular, proposals that did not consider user sentiment were problematic because they made it difficult to motivate actual initiatives and gain the support of stakeholders.

[1104] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1105] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for analyzing the user's emotional data; and means for reflecting the analyzed emotional data in the generative AI model. This makes it possible to generate personalized renovation ideas that reflect region-specific data and the user's emotions.

[1106] "Regional population data" refers to information about the distribution and demographic trends of the population in a specific region.

[1107] "Traffic data" refers to information about traffic flow, congestion levels, and traffic volume in a specific area.

[1108] "Neighborhood facility data" refers to information about the location and characteristics of public facilities, commercial facilities, educational facilities, etc., that exist within a specific area.

[1109] "Human flow data" refers to information about the movement patterns and flow of people in a specific area.

[1110] "Preprocessing" refers to processes such as imputing missing values, removing outliers, and normalizing data in order to prepare it for analysis and model training.

[1111] A "generative AI model" is a model that uses artificial intelligence technology to generate a desired output from specific input data.

[1112] A "renovation idea" is a specific plan or proposal for remodeling or renovating an existing building.

[1113] "Training a generative AI model" refers to the training process of adjusting the parameters of a generative AI model using appropriate input data to obtain highly accurate output.

[1114] "User emotional data" refers to information about the emotions a user is experiencing, such as data on various emotions like anticipation, anxiety, and excitement.

[1115] An "emotion engine" is a system that analyzes the emotions of a user based on the data they input and outputs them in a specific format.

[1116] An "image generation model" is an artificial intelligence model that generates visual materials such as images and videos based on specific input data.

[1117] A "user interface" refers to the means of operation, such as screens and forms, that a user uses to input information into a system.

[1118] "Promotional materials" refer to visual and text content used to advertise a project or proposal, such as generated renovation ideas and design images.

[1119] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user, each with its own role and specific processing as follows.

[1120] Server Processing

[1121] Data Acquisition and Preprocessing

[1122] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from statistical agency APIs, and traffic data is obtained from traffic information services. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. Methods such as imputing missing values ​​with the mean or mode are used. Outlier removal is performed by deleting data that is statistically determined to be outliers. Finally, the data is standardized to shape it into a form suitable for model training.

[1123] Training of Generative AI Models

[1124] The server uses pre-processed data to train a generative AI model to generate optimal utilization ideas for sparsely populated buildings. The deep learning frameworks used include TensorFlow and PyTorch. During training, the dataset is divided into training and validation data, and the model's performance is evaluated and optimized. For example, 80% of the data might be used as training data and 20% as validation data.

[1125] Processing the request

[1126] The server receives detailed information about sparsely populated buildings sent from users via their devices. This information includes the building's location, current usage, and the user's wishes and requirements. This information is then input into an AI model to generate renovation ideas. For example, if the target is a closed school in XX City, Nagano Prefecture, the server would provide information such as its location, current usage, and the user's wishes (e.g., wanting to use it as a community center).

[1127] Emotional engine integration

[1128] The server uses an emotion engine to analyze the user's emotional data. The emotion engine uses NLP (Neuro-Linguistic Programming) techniques to extract the user's emotions from text and reflects that data in the generating AI model. For example, if a user inputs "I have high hopes for this project, but I also have some anxieties," the emotions of "excitement" and "anxiety" will be analyzed.

[1129] Idea generation and promotional material creation

[1130] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the completed project. For example, it uses a GAN (Generative Opposite Network) to generate realistic design images. The generated ideas and design images are used as promotional materials and returned to the terminal.

[1131] Terminal processing

[1132] User Interface

[1133] The device provides users with an interface for entering detailed information about sparsely populated buildings. This might include forms or photo upload functions. Users use this interface to input location information, current photos, desired usage, and their feelings.

[1134] Send Request

[1135] The terminal formats the information entered by the user and sends it to the server. For example, it converts the data to JSON format, performs error checking, and then sends it.

[1136] Displaying Results

[1137] The server returns renovation ideas and promotional materials (design images and videos) to the user. The terminal provides a viewer to display the generated design images and videos, making it easy for the user to review them.

[1138] User processing

[1139] Entering information

[1140] Through the terminal interface, users input information about the building's location, size, current use, and their wishes, conditions, and emotions (excitement, anticipation, anxiety, etc.) regarding the renovation. For example, they might input their wish to "use a closed school in XX City, Nagano Prefecture as a community center," along with photos of the current state and their "expectations and anxieties about the project."

[1141] Idea verification

[1142] Review the proposed renovation ideas and promotional materials. View the generated design images and videos to develop a concrete renovation plan. For example, determine the project's progress based on the design images and videos displayed on the device.

[1143] Example of a prompt

[1144] "We want to use a closed school located in a sparsely populated area as a community center. Please generate renovation ideas based on the following information:

[1145] Building location: Nagano Prefecture, XX City, △△ Town

[1146] Current photos: xxx.jpg, yyy.jpg

[1147] Current use: A closed elementary school

[1148] User emotions: Expectation (want to contribute to regional revitalization), anxiety (worried about whether it will succeed)

[1149] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. Furthermore, the introduction of an emotion engine enables personalized suggestions that take into account the user's emotions.

[1150] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1151] Step 1: Data Collection

[1152] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, it uses APIs from statistical agencies and traffic information services.

[1153] Input: Raw data from external APIs or databases.

[1154] Output: Collected raw data.

[1155] Step 2: Data Preprocessing

[1156] The server performs missing value imputation, outlier removal, and standardization on the collected data. For example, it imputates missing values ​​with the mean, statistically removes outliers, and normalizes numerical data from 0 to 1.

[1157] Input: Collected raw data.

[1158] Output: Pre-processed data.

[1159] Step 3: Training the Generative AI Model

[1160] The server uses pre-processed data to train a generative AI model. For example, it can use TensorFlow or PyTorch to prepare training and validation data and then train the model.

[1161] Input: Pre-processed data.

[1162] Output: A trained generative AI model.

[1163] Step 4: Receiving User Information

[1164] The server receives detailed information about sparsely populated buildings sent from users via their terminals. This includes, for example, the building's location, current usage, and the user's preferences and requirements.

[1165] Input: Building information submitted by the user.

[1166] Output: Detailed information stored on the server.

[1167] Step 5: Generating renovation ideas

[1168] The server inputs the received detailed information into a generating AI model to produce renovation ideas. For example, it proposes the optimal renovation plan based on the building's location and intended use.

[1169] Input: Detailed information and a trained generative AI model.

[1170] Output: Renovation ideas.

[1171] Step 6: Analyzing emotional data

[1172] The server analyzes emotional data from users using an emotion engine. For example, it extracts emotions from text using NLP (Neuro-Linguistic Programming) techniques.

[1173] Input: User sentiment data.

[1174] Output: Analyzed sentiment data.

[1175] Step 7: Personalized Idea Generation

[1176] The server incorporates the analyzed emotional data into a generating AI model to create personalized ideas. It provides renovation ideas that reflect the user's emotions.

[1177] Input: Analyzed sentiment data and detailed information.

[1178] Output: Personalized renovation ideas.

[1179] Step 8: Create promotional materials

[1180] The server uses image generation models to create design images and videos based on the generated renovation ideas. For example, it uses GANs to generate realistic images of the finished product.

[1181] Input: Generated renovation ideas.

[1182] Output: Promotional materials (design images, videos).

[1183] Step 9: Return of Results

[1184] The server returns the generated renovation ideas and promotional materials to the terminal and provides them to the user.

[1185] Input: Promotional material.

[1186] Output: Data returned to the terminal.

[1187] Step 10: User Input

[1188] Users input detailed information such as the building's location, photos, desired usage, and emotions through the device's interface.

[1189] Input: Various data entered by the user.

[1190] Output: Detailed data sent to the terminal.

[1191] Step 11: Submit Request

[1192] The terminal formats the information entered by the user and sends it to the server. For example, it converts the information to JSON format, performs error checking, and then sends it.

[1193] Input: Detailed information entered by the user.

[1194] Output: Request data sent to the server.

[1195] Step 12: Displaying the results

[1196] The terminal displays renovation ideas and promotional materials returned from the server to the user. For example, it provides a viewer that allows users to view generated design images and videos.

[1197] Input: Data returned from the server.

[1198] Output: Data displayed to the user.

[1199] (Application Example 2)

[1200] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1201] It is crucial to effectively renovate buildings in sparsely populated areas and promote regional revitalization. However, what is needed is not just the generation of renovation ideas, but also personalized proposals that take into account the user's feelings. Furthermore, there is a lack of visual means to convey how the generated ideas can be specifically useful. To solve these problems, a system is needed that proposes the optimal utilization methods for sparsely populated buildings, as well as providing optimal renovation ideas and promotional materials based on the user's feelings.

[1202] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1203] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; and means including an emotion engine that analyzes emotion data and works in conjunction with the generative AI model to optimize renovation ideas based on the user's emotions. This makes it possible to provide personalized renovation ideas and concrete visual images that meet the user's wishes and emotions.

[1204] "Regional population data" refers to statistical information such as the number of people residing in a specific area, the number of households, age groups, and gender.

[1205] "Traffic data" refers to information such as traffic volume, congestion levels, and the operating status of public transportation in a specific area.

[1206] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, public facilities, and other similar facilities located in a specific area.

[1207] "People flow data" refers to information such as people's movement patterns, flow conditions, and ability to attract customers in a specific area.

[1208] "Preprocessing" refers to the data organization and editing work required to apply large amounts of data to machine learning models.

[1209] A "generative AI model" refers to an artificial intelligence model that uses machine learning to generate optimal renovation ideas from data.

[1210] "Detailed information" refers to information provided by users, such as the location, size, current usage, and desired conditions or requirements for renovations of sparsely populated buildings.

[1211] "Emotional data" refers to information about a user's psychological state and emotions (e.g., enjoyment, anticipation, anxiety) obtained from user input, facial expressions, voice, and behavior.

[1212] An "emotion engine" refers to a system that analyzes user emotional data and incorporates the results into the generated AI model.

[1213] An "image generation model" refers to a model that uses AI to generate actual images from text and other data.

[1214] "Renovation ideas" refer to specific proposals and design plans for how to remodel and reuse sparsely populated buildings.

[1215] "Personalized suggestions" refer to providing individual renovation ideas optimized based on each user's wishes and feelings.

[1216] "Promotional materials" refer to advertising and public relations content created using generated renovation ideas and design images.

[1217] This invention is a system that generates and provides renovation ideas for buildings in sparsely populated areas, thereby supporting regional revitalization. This system consists of the following three main components:

[1218] server

[1219] The server first collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from the census database, traffic data from the traffic management system, nearby facilities data from the Geographic Information System (GIS), and pedestrian flow data from the mobile data analytics service.

[1220] The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardization. For data preprocessing, the Python pandas library is used, for example.

[1221] Using pre-processed data, the server trains a generative AI model using a deep learning framework (TensorFlow or PyTorch). This model is used to generate optimal utilization ideas for sparsely populated buildings. This allows the model to learn diverse data patterns and produce highly accurate generation results.

[1222] When a user sends detailed information about a sparsely populated building from their device, the server receives this information and inputs it into the generated AI model. This detailed information includes the building's location, current usage, and the user's wishes and conditions regarding renovations.

[1223] Furthermore, the server uses an emotion engine to analyze the user's emotional data and integrates this result with a generating AI model to create more personalized renovation ideas. Based on the analyzed emotional data, the server makes suggestions that take into account the user's expectations and anxieties.

[1224] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the finished product. This allows users to have a concrete visual image of the project. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream.

[1225] terminal

[1226] The terminal provides users with a user interface for entering detailed information about sparsely populated buildings. Through this interface, users can input the building's location, current photos, and their wishes and requirements for renovation. Furthermore, they can also input their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[1227] The information entered by the user is formatted into the appropriate format by the terminal and sent to the server. The terminal also performs error checking on the information to ensure that the data is transmitted accurately.

[1228] Renovation ideas and promotional materials (images and videos) returned from the server are displayed on the device. This allows users to create concrete renovation plans based on these materials.

[1229] User

[1230] Users input detailed information and sentimental data about sparsely populated buildings they are interested in via their devices. For example, they input the building's location, size, and current use, as well as their wishes, conditions, and feelings regarding renovations (e.g., excitement, anticipation, anxiety).

[1231] Users can view renovation ideas and promotional materials displayed on their devices and use them to develop concrete project plans. The generated materials can also be used on social media and in presentations, serving as valuable tools to attract the attention of local communities and stakeholders.

[1232] Specific example

[1233] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. The user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project) from their device.

[1234] The server receives this information and generates renovation ideas using a generative AI model. The emotion engine analyzes the user's emotional data and incorporates the results into the generative AI model, providing personalized ideas that take the user's emotions into consideration.

[1235] Example of a prompt

[1236] User input:

[1237] Building details: Abandoned school in Nagano Prefecture, area 1200m² 2 Currently used as a warehouse.

[1238] Renovation request: I would like to use it as a community center.

[1239] Emotions: I'm very excited, but also a little anxious.

[1240] Expected output:

[1241] Generated from AI model: "By renovating the abandoned school into a community center, local interaction can be deepened and it can be used as a new tourist attraction. In particular, it is important to incorporate a design that prioritizes comfort. Since users have given positive feedback, let's give maximum consideration to comfort and functionality."

[1242] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[1243] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1244] Step 1:

[1245] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. For example, population data is obtained from a census database, and traffic data is obtained from a traffic management system. The input is raw data obtained from APIs and databases, and the output is preprocessed, formatted data.

[1246] Step 2:

[1247] The server uses preprocessed data to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). This allows the server to build a generative AI model for generating optimal utilization ideas for sparsely populated buildings. The input is preprocessed data, and the output is the trained generative AI model. Specifically, the model is trained using training data, and its performance is evaluated.

[1248] Step 3:

[1249] Users input detailed information about sparsely populated buildings and their own feelings using a user interface on their device. This might include the building's location, current photos, renovation hopes and conditions, and emotions (e.g., excitement, anticipation, anxiety). The input is information provided by the user, and the output is formatted detailed data.

[1250] Step 4:

[1251] The terminal formats the user-entered details into the appropriate format and sends them to the server. It also performs error checking to ensure the data is sent accurately to the server. The input is the information entered by the user, and the output is the formatted data after error checking.

[1252] Step 5:

[1253] The server receives detailed information sent by the user and inputs this information into the generating AI model. Specifically, it supplies the AI ​​model with information such as the building's location, usage, and the user's wishes and conditions. The input is the detailed information received from the user, and the output is the generated renovation ideas.

[1254] Step 6:

[1255] The server analyzes emotional data and works with a generative AI model to optimize renovation ideas based on the user's emotions. The emotion engine analyzes user input, facial expressions, voice, etc., and provides emotion-based data. The input is the user's emotional data, and the output is an optimized renovation idea based on those emotions.

[1256] Step 7:

[1257] The server uses an image generation model to generate images and videos of the completed design based on the generated renovation ideas. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream. The input is the generated renovation idea, and the output is the images and videos of the completed design.

[1258] Step 8:

[1259] The server returns the generated renovation ideas and design images to the terminal. This allows the user to have a concrete visual image. The input is the generated renovation ideas and design images, and the output is the data returned to the user.

[1260] Step 9:

[1261] The terminal displays renovation ideas and promotional materials (images and videos) returned from the server to the user. This allows the user to create a concrete renovation plan based on these materials. The input is data returned from the server, and the output is the renovation ideas and design images displayed to the user.

[1262] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1263] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1264] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1265] [Fourth Embodiment]

[1266] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1267] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1268] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1269] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1270] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1271] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1272] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1273] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1274] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1275] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1276] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1277] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1278] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1279] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[1280] server

[1281] Data Acquisition and Preprocessing

[1282] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. It then performs preprocessing on the collected data, such as imputing missing values, removing outliers, and standardizing the data. This prepares the data for training.

[1283] Training of Generative AI Models

[1284] The server trains a generative AI model using preprocessed data. This AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used for training, and the data is divided into training and validation data for model training. Model evaluation is also performed at this stage to verify its performance.

[1285] Processing the request

[1286] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. This information includes the building's location and current usage. The server inputs this information into an AI model to generate ideas for its utilization.

[1287] Idea generation and promotional material creation

[1288] Based on the generated utilization ideas, the server uses an image generation model to create images and videos that visually represent the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated utilization ideas and promotional materials are then returned to the terminal.

[1289] terminal

[1290] User Interface

[1291] The terminal provides users with an interface for entering detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, and their wishes and conditions regarding renovations.

[1292] Send Request

[1293] To send the information entered by the user to the server, the terminal formats the data and sends the request.

[1294] Displaying Results

[1295] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[1296] User

[1297] Entering information

[1298] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This includes, for example, the building's location, size, and current use. Users can also enter their wishes and conditions regarding renovations.

[1299] Idea verification

[1300] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[1301] Specific example

[1302] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information of the vacant school building, its current condition (with photos), and their desired use (for example, to use it as a community center) through the terminal interface.

[1303] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated space. These results are returned to the terminal, allowing the user to review them and develop a concrete renovation plan.

[1304] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[1305] The following describes the processing flow.

[1306] Step 1:

[1307] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed regularly with the aim of obtaining accurate and up-to-date information.

[1308] Step 2:

[1309] The server performs preprocessing on the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. This prepares a dataset suitable for subsequent AI model training.

[1310] Step 3:

[1311] The server trains a generative AI model using preprocessed data. It splits the data into training data and validation data, trains the model using a deep learning framework (e.g., TensorFlow or PyTorch), and evaluates the model's performance.

[1312] Step 4:

[1313] Users input detailed information about sparsely populated buildings (location, current photos, renovation requests and conditions, etc.) through the terminal's user interface. Once the input is complete, they submit a request for renovation idea generation.

[1314] Step 5:

[1315] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[1316] Step 6:

[1317] The server receives detailed information about sparsely populated buildings sent from the terminal. The received information includes the building's location and details on how to use it.

[1318] Step 7:

[1319] The server inputs the received information into a generating AI model, which then generates optimal renovation ideas for sparsely populated buildings. This AI model proposes the best plan based on pre-trained data.

[1320] Step 8:

[1321] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. This provides users and stakeholders with concrete visual images.

[1322] Step 9:

[1323] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is then formatted to be useful to the user.

[1324] Step 10:

[1325] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[1326] Step 11:

[1327] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[1328] (Example 1)

[1329] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1330] There is a problem in sparsely populated areas where the utilization of buildings is not progressing, leading to a stagnation of regional revitalization. Furthermore, current renovation plans are not concrete, making it difficult to attract the interest of stakeholders and reach a consensus for implementation. In this situation, there is a need for a system that can quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images to support the realization of renovation plans.

[1331] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1332] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for generating post-renovation design images and videos as promotional materials; means for providing users with concrete visual images to help them plan renovations; and means for providing the generated products as materials for social media and presentations. This makes it possible to quickly generate optimal utilization ideas for sparsely populated buildings and provide concrete visual images.

[1333] "Regional population data" refers to information about the number of people living in a specific area, population density, age distribution, and so on.

[1334] "Traffic data" refers to information regarding traffic volume, the number of traffic accidents, and the types of vehicles traveling in a specific area.

[1335] "Nearby facilities data" refers to information about the location, number, and types of schools, hospitals, supermarkets, public facilities, etc., that exist in a specific area.

[1336] "Human flow data" refers to information that shows the movement patterns and flow of people in a specific area.

[1337] "Preprocessing" refers to the process of preparing collected data into a format that is easy to analyze, such as imputing missing values, removing outliers, and standardizing the data.

[1338] A "generative AI model" refers to an artificial intelligence model that uses machine learning and deep learning techniques to generate useful information and ideas from new data.

[1339] "Detailed information on sparsely populated buildings from users" refers to information including location information, current usage status, and desired renovations for buildings in sparsely populated areas where renovation is being considered.

[1340] "Renovation ideas" refer to ideas that use generative AI models to suggest optimal ways to utilize or renovate sparsely populated buildings.

[1341] An "image generation model" refers to a model that generates visual design images and videos based on ideas proposed by a generative AI model.

[1342] "Promotional materials" refer to images and videos that visually demonstrate the design and usage of a renovated building, and are intended to attract the attention of project stakeholders and the general public.

[1343] "Visual images" refer to images or videos that visually embody the generated renovation ideas.

[1344] "Materials for social media and presentations" refers to digital files such as images and videos used in social media posts and presentations to promote renovation ideas.

[1345] This invention is a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization. This system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each are described below.

[1346] server

[1347] Data Acquisition and Preprocessing

[1348] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data. Specifically, it obtains this data from external APIs (e.g., location APIs and traffic APIs) and databases (e.g., local databases). The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardizing the data. Software tools such as the Python Pandas library are used for preprocessing.

[1349] Training of Generative AI Models

[1350] The server trains a generative AI model using pre-processed data. The generative AI model is designed to generate optimal utilization ideas for sparsely populated buildings. Deep learning frameworks such as TensorFlow and PyTorch are used to train the model, and the data is split into training and validation sets for training.

[1351] Receiving user requests

[1352] The server receives detailed information about sparsely populated buildings entered by the user from their device. This information includes the building's location, current usage, and renovation requests and conditions. This data is sent to the server via an HTTP request.

[1353] Idea generation using AI models

[1354] The server inputs the received detailed information into a generating AI model to generate optimal renovation ideas. Based on the input data, the generating AI model proposes specific ways to utilize the property.

[1355] Creating promotional materials

[1356] The server uses an image generation model to create design images and videos of the completed project based on the generated renovation ideas. This promotional material visually showcases the generated ideas and utilizes image generation technologies such as DeepArt and MidJourney.

[1357] Return of renovation ideas

[1358] The server returns the generated renovation ideas and design images to the user, allowing them to develop a concrete renovation plan.

[1359] terminal

[1360] User Interface

[1361] The terminal provides an interface that allows users to input detailed information about sparsely populated buildings. Through this interface, users can enter details such as the building's location, current photos, and desired renovations.

[1362] Send Request

[1363] The terminal sends the user's entered details to the server. The transmission occurs via an HTTP request, and the data is formatted into the appropriate format.

[1364] Displaying Results

[1365] The terminal displays renovation ideas and design images returned from the server to the user, providing information to help them move forward with concrete project planning.

[1366] User

[1367] Entering information

[1368] Users enter detailed information about sparsely populated buildings they are interested in through the terminal's interface. This information includes the building's location, current photos, and their wishes and requirements regarding renovations.

[1369] Idea verification

[1370] Users will view renovation ideas and design images displayed on their devices and use them to create concrete renovation plans. They will also use the generated promotional materials on social media and in presentations to attract the attention of stakeholders.

[1371] Examples of specific cases and prompt statements

[1372] For example, if the user were to use this system to generate renovation ideas for a vacant school building located in a sparsely populated area, they would enter the following prompt:

[1373] "The location of the vacant school building is [latitude and longitude]. Its current use is [description]. The desired use after renovation is a community center. Please generate the most suitable utilization ideas."

[1374] The server receives this information and uses a generative AI model to generate optimal utilization ideas. Then, an image generation model creates design images and videos of the renovated space, and these results are returned to the user's device. The user can then review them and proceed with a concrete renovation plan.

[1375] This will enable the effective support of regional revitalization through a system that generates renovation ideas for sparsely populated buildings using a generation AI model.

[1376] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1377] Step 1: Data Collection

[1378] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. Specifically, the server uses location APIs and traffic APIs to obtain data for a specified area. For example, it obtains local population data from a census API and traffic data from a traffic API. Input requires an API request, and output is data returned in JSON format.

[1379] Step 2: Data Preprocessing

[1380] The server preprocesses the collected data. Specifically, it uses the Python Pandas library to impute missing values, remove outliers, and standardize the data. The data collected in step 1 is used as input, and the output is clean, preprocessed data. This is to prepare the data for training the generative AI model.

[1381] Step 3: Training the Generative AI Model

[1382] The server uses preprocessed data to train a generative AI model. Specifically, it trains a deep learning model using TensorFlow or PyTorch. Preprocessed data is used as input, and the trained generative AI model is obtained as output. Furthermore, the model is evaluated at this stage to confirm its performance.

[1383] Step 4: Receiving User Requests

[1384] The terminal sends detailed information about sparsely populated buildings entered by the user to the server. Specifically, it sends information obtained through the user interface as an HTTP request. The input is information entered by the user (location information, current photos, renovation requests), and the output is received by the server.

[1385] Step 5: Idea generation using AI models

[1386] The server inputs the received user details into a generating AI model to generate optimal renovation ideas. Specifically, the generating AI model proposes utilization methods based on the input data. The input uses detailed information received from the user, and the output is the generated renovation ideas.

[1387] Step 6: Create promotional materials

[1388] The server uses an image generation model to create design images and videos based on the generated renovation ideas. Specifically, it utilizes tools such as DeepArt and MidJourney to generate visuals of the renovated area. The generated renovation ideas are used as input, and promotional materials in image and video formats are obtained as output.

[1389] Step 7: Return of renovation ideas

[1390] The server returns the generated renovation ideas and promotional materials to the terminal. Specifically, it formats the generated content and sends it to the terminal as an HTTP response. The generated content (ideas and materials) is used as input, and the output is sent back to the terminal.

[1391] Step 8: Displaying the results

[1392] The terminal displays renovation ideas and promotional materials returned from the server to the user. Specifically, it displays the returned data on the user interface and formats it so that the user can review it. The input is the data returned from the server, and the output is displayed in a format that the user can view. The user then uses this to create a concrete renovation plan.

[1393] (Application Example 1)

[1394] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1395] Renovating buildings in sparsely populated areas is difficult due to population decline and decreased demand. Furthermore, the process of proposing concrete renovation ideas and designs is cumbersome, highlighting the need for effective methods of regional revitalization. Therefore, a system is needed to quickly and efficiently generate and provide renovation ideas and concrete designs for physical stores in sparsely populated areas.

[1396] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1397] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; and means for learning a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data. This makes it possible to quickly and efficiently generate renovation ideas for physical stores located in sparsely populated areas and provide users with design images and videos based on those ideas.

[1398] The server also includes means for receiving detailed information about sparsely populated buildings from users, means for inputting the received detailed information into a generation AI model to generate renovation ideas, means for generating completed design images and videos using an image generation model based on the generated ideas, and means for returning the generated renovation ideas and design images and videos to the user via a smartphone application. This allows users to easily check specific renovation ideas and their visual images, and use them to plan for regional revitalization.

[1399] "Regional population data" refers to information about the population size, population density, age distribution, etc., within a specific region.

[1400] "Traffic data" refers to information regarding traffic volume, congestion, and the operating status of public transportation within a specific area.

[1401] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, entertainment facilities, and other facilities located within a specific area.

[1402] "People flow data" refers to information about the movement and dwell time of people within a specific area, as well as the number of visitors and their travel routes.

[1403] "Preprocessing" refers to the process of preparing collected data for training AI models by performing actions such as imputing missing values, removing outliers, and standardizing the data.

[1404] A "generative AI model" is an artificial intelligence model that learns from collected and pre-processed data to generate optimal utilization ideas for sparsely populated buildings.

[1405] "Detailed information on underutilized buildings from users" refers to specific information such as the location of the underutilized building, its current use, size, photos, and any wishes or conditions regarding renovation.

[1406] An "image generation model" is an artificial intelligence model that generates design images and videos of a renovated space based on ideas generated by a generative AI model.

[1407] A "smartphone application" is application software that runs on a smartphone and allows users to input detailed information about renovations and view renovation ideas, design images, and videos returned from a server.

[1408] "Promotional materials" refer to the generated renovation ideas and the design images and videos based on them, which will be used for promotional activities aimed at revitalizing the local community.

[1409] This invention is a system that utilizes generative AI models to renovate buildings in sparsely populated areas and support regional revitalization. The system is divided into three main components: a server, terminals, and users.

[1410] server

[1411] Data Acquisition and Preprocessing

[1412] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This data is preprocessed, including imputing missing values, removing outliers, and standardization, to prepare it as training data. Data processing libraries such as Python and Pandas are used for collection and preprocessing.

[1413] Training of Generative AI Models

[1414] The server trains a generative AI model using preprocessed data. Deep learning frameworks such as TensorFlow and PyTorch are used to train the AI ​​model. This AI model generates optimal utilization ideas for sparsely populated buildings. During the training process, the data is divided into training data and validation data, and the model's performance is evaluated.

[1415] Processing the request

[1416] The server receives detailed information about sparsely populated buildings sent by users via their devices. This includes the building's location, current usage, and any wishes or conditions for renovation. The server inputs the received information into an AI model to generate renovation ideas.

[1417] Idea generation and promotional material creation

[1418] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. Technologies such as Generative Opposite Networks (GANs) are used for image generation. The generated renovation ideas and design images and videos are returned to the user via a smartphone application.

[1419] terminal

[1420] User Interface

[1421] The terminal provides users with an interface for inputting detailed information about sparsely populated buildings. Users can use the terminal to input information such as the building's location, current photos, and renovation requests. This allows users to easily send the necessary information to the server.

[1422] Send Request

[1423] The terminal formats the information entered by the user into a data format and sends it to the server. The HTTPS protocol is used to send requests, ensuring secure and efficient data exchange.

[1424] Displaying Results

[1425] The server returns renovation ideas and promotional materials (design images and videos) to the user. This allows the user to create a concrete renovation plan.

[1426] User

[1427] Entering information

[1428] Users input detailed information about sparsely populated buildings they are interested in through the interface of a smartphone application. This includes, for example, the building's location, size, current use, photos, and their wishes and requirements for renovation.

[1429] Idea verification

[1430] Users can review renovation ideas and promotional materials displayed on their devices. Based on the suggested ideas, users can proceed with developing concrete project plans. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[1431] Specific example

[1432] For example, consider a scenario where this system is used to generate renovation ideas for a vacant school building located in a sparsely populated area. The user inputs the location information, current condition (with photos), and desired use (e.g., to use it as a community center) of the vacant school building through a smartphone application interface. The server receives this information and uses a generation AI model to generate optimal utilization ideas. Based on the generated ideas, an image generation model creates design images and videos of the renovated building. These results are returned to the user via the smartphone application, allowing the user to review them and develop a concrete renovation plan.

[1433] Example of a prompt

[1434] "This building is currently vacant. We would like to renovate it and turn it into a shop selling local specialty products. Please generate a renovation plan for the building, create an image rendering and a promotional video. We have attached some reference photos."

[1435] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization.

[1436] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1437] Step 1:

[1438] Data collection (server)

[1439] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. This data collection is performed using the Python requests library, and database management systems such as SQLite and PostgreSQL are used for queries from the databases. Inputs are external API endpoints and database queries, and outputs are the respective data.

[1440] Step 2:

[1441] Data preprocessing (server)

[1442] The collected data is preprocessed. This preprocessing includes imputing missing values, removing outliers, and standardization. The program uses the Pandas library to perform data cleaning. Specifically, missing values ​​are imputed with the median, and outliers (e.g., data exceeding ±3σ) are removed. The input is the collected raw data, and the output is the preprocessed data.

[1443] Step 3:

[1444] Training of generative AI models (server)

[1445] The server trains a generative AI model using preprocessed data. It builds and trains the model using deep learning frameworks such as TensorFlow and PyTorch. The data is divided into training and validation data, and the model's performance is evaluated throughout the training process. The input is preprocessed data, and the output is the trained generative AI model.

[1446] Step 4:

[1447] Receiving user input information (terminal)

[1448] The terminal provides a user interface, allowing users to input detailed information about sparsely populated buildings. This information includes the building's location, current use, size, photos, and any desired renovations or requirements. The input is the building information entered by the user, and the output is the data organized for transmission to the server.

[1449] Step 5:

[1450] Sending a request (from the terminal)

[1451] The terminal organizes the information entered by the user and generates a request to send to the server. The request contains all the collected information in JSON format. The input is the building information entered by the user, and the output is the request data sent to the server.

[1452] Step 6:

[1453] Renovation idea generation (server)

[1454] The server inputs detailed information received from the user into a generating AI model to generate renovation ideas. During this process, prompt statements are created and input into the model. The input is the user's building information, and the output is the generated renovation ideas.

[1455] Step 7:

[1456] Design image and video generation (server)

[1457] Based on the generated renovation ideas, the server uses an image generation model to produce images and videos of the completed design. This process employs generative opposite-party network (GAN) technology. The input is the renovation idea, and the output is the generated images and videos.

[1458] Step 8:

[1459] Return of results (server)

[1460] The generated renovation ideas, design images, and videos are returned to the user via a smartphone application. The input is design images and videos, and the output is the result displayed on the user's device.

[1461] Step 9:

[1462] Checking the results (user)

[1463] Users view renovation ideas and promotional materials (images and videos) displayed on their devices. Input is information returned from the server, and output is the visual content the user reviews. This allows for the creation of concrete renovation plans.

[1464] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1465] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user. The roles and specific processes of each component are described below.

[1466] server

[1467] Data Acquisition and Preprocessing

[1468] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. The collected data is preprocessed, including imputation of missing values, removal of outliers, and standardization, to prepare it for training AI models.

[1469] Training of Generative AI Models

[1470] The server uses pre-processed data to train a generative AI model that generates optimal utilization ideas for sparsely populated buildings. Using a deep learning framework (e.g., TensorFlow or PyTorch), the model is trained on training data and validation data, and its performance is evaluated.

[1471] Processing the request

[1472] When a user sends detailed information about a sparsely populated building via their device, the server receives that information. The received information includes the building's location, current usage, and the user's wishes and requirements. The server inputs this detailed information into a generating AI model to generate renovation ideas.

[1473] Emotional engine integration

[1474] The emotion engine analyzes the user's emotional data and works in conjunction with a generative AI model to optimize renovation idea recommendations. Based on the information entered by the user and the emotional data analyzed by the emotion engine, more personalized renovation ideas are generated.

[1475] Idea generation and promotional material creation

[1476] Based on the generated renovation ideas, the server uses an image generation model to create images and videos of the completed design. This provides users and stakeholders with concrete visual images and creates materials that can be used for promotion. The generated ideas and promotional materials are then returned to the terminal.

[1477] terminal

[1478] User Interface

[1479] The device provides users with an interface for inputting detailed information about sparsely populated buildings. Through this interface, users can input location information, current photos, renovation wishes and conditions, and even their own feelings.

[1480] Send Request

[1481] The terminal formats the information entered by the user and sends it to the server. The terminal performs data formatting and error checking to ensure that the information is transmitted accurately.

[1482] Displaying Results

[1483] The server returns renovation ideas and promotional materials (images and videos) to the user. This allows the user to develop a concrete renovation plan based on the suggested ideas.

[1484] User

[1485] Entering information

[1486] Through the terminal interface, users can input detailed information and emotional data about sparsely populated buildings that interest them. For example, they can input the building's location, size, and current use, as well as their wishes and conditions regarding renovations, and their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[1487] Idea verification

[1488] Users can review renovation ideas and promotional materials displayed on their devices and proceed with concrete project plans based on the proposed ideas. Furthermore, the generated materials can be used on social media and in presentations to attract the attention of local communities and stakeholders.

[1489] Specific example

[1490] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. Through the terminal interface, the user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project).

[1491] The server receives this information and uses a generative AI model to generate optimal renovation ideas. The emotion engine analyzes the user's emotional data and reflects the results in the generative AI model, providing personalized ideas that take the user's emotions into consideration. Based on the generated ideas, the image generation model creates design images and videos of the renovated space. These results are returned to the device, allowing the user to review them and develop a concrete plan.

[1492] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[1493] The following describes the processing flow.

[1494] Step 1:

[1495] The server collects local population data, traffic data, nearby facility data, and pedestrian flow data from external APIs and databases. This allows for accurate and up-to-date information about sparsely populated areas.

[1496] Step 2:

[1497] The server preprocesses the collected data. Specifically, it performs tasks such as imputing missing values, removing outliers, and standardizing the data. In this way, it prepares the data in a format suitable for training.

[1498] Step 3:

[1499] The server uses pre-processed data to train a generative AI model. A deep learning framework (such as TensorFlow or PyTorch) is used for training, dividing the data into training and validation sets to train the model, and then its performance is evaluated.

[1500] Step 4:

[1501] Users input detailed information about sparsely populated buildings through the terminal's user interface. This information includes location data, current photos, renovation wishes and requirements, and even emotional data (e.g., excitement, anticipation, anxiety).

[1502] Step 5:

[1503] The terminal formats the information entered by the user and sends it to the server. The terminal performs formatting and error checking to ensure that the data is transmitted accurately.

[1504] Step 6:

[1505] The server receives detailed information and sentiment data about sparsely populated buildings sent from the terminal. The received information includes the building's location and details about its usage.

[1506] Step 7:

[1507] The server inputs the received detailed information and emotional data into the generating AI model and emotion engine to generate optimal renovation ideas for sparsely populated buildings. In this process, the emotion engine analyzes the user's emotions and reflects the results in the generating AI model.

[1508] Step 8:

[1509] The server uses an image generation model based on the generated renovation ideas to create design images and videos of the completed project. The generated designs are mindful of the user's emotions.

[1510] Step 9:

[1511] The server returns the generated renovation ideas and promotional materials to the terminal. The returned data is formatted into a user-friendly format.

[1512] Step 10:

[1513] The terminal displays renovation ideas, images, and videos returned from the server to the user. The user can review these and develop a concrete renovation plan.

[1514] Step 11:

[1515] Users use renovation ideas and promotional materials displayed on their devices to create attention-grabbing presentations and social media posts, thereby attracting the interest of local communities and stakeholders.

[1516] (Example 2)

[1517] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1518] In the renovation of buildings in sparsely populated areas, conventional methods failed to effectively utilize region-specific data, and it was difficult to provide personalized proposals that reflected user sentiment. This made it difficult to generate appropriate utilization ideas and to realize concrete plans that would contribute to regional revitalization. In particular, proposals that did not consider user sentiment were problematic because they made it difficult to motivate actual initiatives and gain the support of stakeholders.

[1519] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1520] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; means for analyzing the user's emotional data; and means for reflecting the analyzed emotional data in the generative AI model. This makes it possible to generate personalized renovation ideas that reflect region-specific data and the user's emotions.

[1521] "Regional population data" refers to information about the distribution and demographic trends of the population in a specific region.

[1522] "Traffic data" refers to information about traffic flow, congestion levels, and traffic volume in a specific area.

[1523] "Neighborhood facility data" refers to information about the location and characteristics of public facilities, commercial facilities, educational facilities, etc., that exist within a specific area.

[1524] "Human flow data" refers to information about the movement patterns and flow of people in a specific area.

[1525] "Preprocessing" refers to processes such as imputing missing values, removing outliers, and normalizing data in order to prepare it for analysis and model training.

[1526] A "generative AI model" is a model that uses artificial intelligence technology to generate a desired output from specific input data.

[1527] A "renovation idea" is a specific plan or proposal for remodeling or renovating an existing building.

[1528] "Training a generative AI model" refers to the training process of adjusting the parameters of a generative AI model using appropriate input data to obtain highly accurate output.

[1529] "User emotional data" refers to information about the emotions a user is experiencing, such as data on various emotions like anticipation, anxiety, and excitement.

[1530] An "emotion engine" is a system that analyzes the emotions of a user based on the data they input and outputs them in a specific format.

[1531] An "image generation model" is an artificial intelligence model that generates visual materials such as images and videos based on specific input data.

[1532] A "user interface" refers to the means of operation, such as screens and forms, that a user uses to input information into a system.

[1533] "Promotional materials" refer to visual and text content used to advertise a project or proposal, such as generated renovation ideas and design images.

[1534] This invention combines a system that utilizes generative AI to renovate buildings in sparsely populated areas and support regional revitalization with an emotion engine that recognizes user emotions. The system consists of three main components: a server, a terminal, and a user, each with its own role and specific processing as follows.

[1535] Server Processing

[1536] Data Acquisition and Preprocessing

[1537] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from statistical agency APIs, and traffic data is obtained from traffic information services. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. Methods such as imputing missing values ​​with the mean or mode are used. Outlier removal is performed by deleting data that is statistically determined to be outliers. Finally, the data is standardized to shape it into a form suitable for model training.

[1538] Training of Generative AI Models

[1539] The server uses pre-processed data to train a generative AI model to generate optimal utilization ideas for sparsely populated buildings. The deep learning frameworks used include TensorFlow and PyTorch. During training, the dataset is divided into training and validation data, and the model's performance is evaluated and optimized. For example, 80% of the data might be used as training data and 20% as validation data.

[1540] Processing the request

[1541] The server receives detailed information about sparsely populated buildings sent from users via their devices. This information includes the building's location, current usage, and the user's wishes and requirements. This information is then input into an AI model to generate renovation ideas. For example, if the target is a closed school in XX City, Nagano Prefecture, the server would provide information such as its location, current usage, and the user's wishes (e.g., wanting to use it as a community center).

[1542] Emotional engine integration

[1543] The server uses an emotion engine to analyze the user's emotional data. The emotion engine uses NLP (Neuro-Linguistic Programming) techniques to extract the user's emotions from text and reflects that data in the generating AI model. For example, if a user inputs "I have high hopes for this project, but I also have some anxieties," the emotions of "excitement" and "anxiety" will be analyzed.

[1544] Idea generation and promotional material creation

[1545] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the completed project. For example, it uses a GAN (Generative Opposite Network) to generate realistic design images. The generated ideas and design images are used as promotional materials and returned to the terminal.

[1546] Terminal processing

[1547] User Interface

[1548] The device provides users with an interface for entering detailed information about sparsely populated buildings. This might include forms or photo upload functions. Users use this interface to input location information, current photos, desired usage, and their feelings.

[1549] Send Request

[1550] The terminal formats the information entered by the user and sends it to the server. For example, it converts the data to JSON format, performs error checking, and then sends it.

[1551] Displaying Results

[1552] The server returns renovation ideas and promotional materials (design images and videos) to the user. The terminal provides a viewer to display the generated design images and videos, making it easy for the user to review them.

[1553] User processing

[1554] Entering information

[1555] Through the terminal interface, users input information about the building's location, size, current use, and their wishes, conditions, and emotions (excitement, anticipation, anxiety, etc.) regarding the renovation. For example, they might input their wish to "use a closed school in XX City, Nagano Prefecture as a community center," along with photos of the current state and their "expectations and anxieties about the project."

[1556] Idea verification

[1557] Review the proposed renovation ideas and promotional materials. View the generated design images and videos to develop a concrete renovation plan. For example, determine the project's progress based on the design images and videos displayed on the device.

[1558] Example of a prompt

[1559] "We want to use a closed school located in a sparsely populated area as a community center. Please generate renovation ideas based on the following information:

[1560] Building location: Nagano Prefecture, XX City, △△ Town

[1561] Current photos: xxx.jpg, yyy.jpg

[1562] Current use: A closed elementary school

[1563] User emotions: Expectation (want to contribute to regional revitalization), anxiety (worried about whether it will succeed)

[1564] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. Furthermore, the introduction of an emotion engine enables personalized suggestions that take into account the user's emotions.

[1565] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1566] Step 1: Data Collection

[1567] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, it uses APIs from statistical agencies and traffic information services.

[1568] Input: Raw data from external APIs or databases.

[1569] Output: Collected raw data.

[1570] Step 2: Data Preprocessing

[1571] The server performs missing value imputation, outlier removal, and standardization on the collected data. For example, it imputates missing values ​​with the mean, statistically removes outliers, and normalizes numerical data from 0 to 1.

[1572] Input: Collected raw data.

[1573] Output: Pre-processed data.

[1574] Step 3: Training the Generative AI Model

[1575] The server uses pre-processed data to train a generative AI model. For example, it can use TensorFlow or PyTorch to prepare training and validation data and then train the model.

[1576] Input: Pre-processed data.

[1577] Output: A trained generative AI model.

[1578] Step 4: Receiving User Information

[1579] The server receives detailed information about sparsely populated buildings sent from users via their terminals. This includes, for example, the building's location, current usage, and the user's preferences and requirements.

[1580] Input: Building information submitted by the user.

[1581] Output: Detailed information stored on the server.

[1582] Step 5: Generating renovation ideas

[1583] The server inputs the received detailed information into a generating AI model to produce renovation ideas. For example, it proposes the optimal renovation plan based on the building's location and intended use.

[1584] Input: Detailed information and a trained generative AI model.

[1585] Output: Renovation ideas.

[1586] Step 6: Analyzing emotional data

[1587] The server analyzes emotional data from users using an emotion engine. For example, it extracts emotions from text using NLP (Neuro-Linguistic Programming) techniques.

[1588] Input: User sentiment data.

[1589] Output: Analyzed sentiment data.

[1590] Step 7: Personalized Idea Generation

[1591] The server incorporates the analyzed emotional data into a generating AI model to create personalized ideas. It provides renovation ideas that reflect the user's emotions.

[1592] Input: Analyzed sentiment data and detailed information.

[1593] Output: Personalized renovation ideas.

[1594] Step 8: Create promotional materials

[1595] The server uses image generation models to create design images and videos based on the generated renovation ideas. For example, it uses GANs to generate realistic images of the finished product.

[1596] Input: Generated renovation ideas.

[1597] Output: Promotional materials (design images, videos).

[1598] Step 9: Return of Results

[1599] The server returns the generated renovation ideas and promotional materials to the terminal and provides them to the user.

[1600] Input: Promotional material.

[1601] Output: Data returned to the terminal.

[1602] Step 10: User Input

[1603] Users input detailed information such as the building's location, photos, desired usage, and emotions through the device's interface.

[1604] Input: Various data entered by the user.

[1605] Output: Detailed data sent to the terminal.

[1606] Step 11: Submit Request

[1607] The terminal formats the information entered by the user and sends it to the server. For example, it converts the information to JSON format, performs error checking, and then sends it.

[1608] Input: Detailed information entered by the user.

[1609] Output: Request data sent to the server.

[1610] Step 12: Displaying the results

[1611] The terminal displays renovation ideas and promotional materials returned from the server to the user. For example, it provides a viewer that allows users to view generated design images and videos.

[1612] Input: Data returned from the server.

[1613] Output: Data displayed to the user.

[1614] (Application Example 2)

[1615] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1616] It is crucial to effectively renovate buildings in sparsely populated areas and promote regional revitalization. However, what is needed is not just the generation of renovation ideas, but also personalized proposals that take into account the user's feelings. Furthermore, there is a lack of visual means to convey how the generated ideas can be specifically useful. To solve these problems, a system is needed that proposes the optimal utilization methods for sparsely populated buildings, as well as providing optimal renovation ideas and promotional materials based on the user's feelings.

[1617] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1618] In this invention, the server includes means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data; means for preprocessing the collected data; means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using the preprocessed data; means for receiving detailed information about sparsely populated buildings from users; means for inputting the received detailed information into the generative AI model to generate renovation ideas; means for generating completed design images using an image generation model based on the generated ideas; means for returning the generated renovation ideas and design images to the user; and means including an emotion engine that analyzes emotion data and works in conjunction with the generative AI model to optimize renovation ideas based on the user's emotions. This makes it possible to provide personalized renovation ideas and concrete visual images that meet the user's wishes and emotions.

[1619] "Regional population data" refers to statistical information such as the number of people residing in a specific area, the number of households, age groups, and gender.

[1620] "Traffic data" refers to information such as traffic volume, congestion levels, and the operating status of public transportation in a specific area.

[1621] "Nearby facilities data" refers to information about commercial facilities, educational institutions, medical facilities, public facilities, and other similar facilities located in a specific area.

[1622] "People flow data" refers to information such as people's movement patterns, flow conditions, and ability to attract customers in a specific area.

[1623] "Preprocessing" refers to the data organization and editing work required to apply large amounts of data to machine learning models.

[1624] A "generative AI model" refers to an artificial intelligence model that uses machine learning to generate optimal renovation ideas from data.

[1625] "Detailed information" refers to information provided by users, such as the location, size, current usage, and desired conditions or requirements for renovations of sparsely populated buildings.

[1626] "Emotional data" refers to information about a user's psychological state and emotions (e.g., enjoyment, anticipation, anxiety) obtained from user input, facial expressions, voice, and behavior.

[1627] An "emotion engine" refers to a system that analyzes user emotional data and incorporates the results into the generated AI model.

[1628] An "image generation model" refers to a model that uses AI to generate actual images from text and other data.

[1629] "Renovation ideas" refer to specific proposals and design plans for how to remodel and reuse sparsely populated buildings.

[1630] "Personalized suggestions" refer to providing individual renovation ideas optimized based on each user's wishes and feelings.

[1631] "Promotional materials" refer to advertising and public relations content created using generated renovation ideas and design images.

[1632] This invention is a system that generates and provides renovation ideas for buildings in sparsely populated areas, thereby supporting regional revitalization. This system consists of the following three main components:

[1633] server

[1634] The server first collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. For example, population data is obtained from the census database, traffic data from the traffic management system, nearby facilities data from the Geographic Information System (GIS), and pedestrian flow data from the mobile data analytics service.

[1635] The collected data undergoes preprocessing, such as imputing missing values, removing outliers, and standardization. For data preprocessing, the Python pandas library is used, for example.

[1636] Using pre-processed data, the server trains a generative AI model using a deep learning framework (TensorFlow or PyTorch). This model is used to generate optimal utilization ideas for sparsely populated buildings. This allows the model to learn diverse data patterns and produce highly accurate generation results.

[1637] When a user sends detailed information about a sparsely populated building from their device, the server receives this information and inputs it into the generated AI model. This detailed information includes the building's location, current usage, and the user's wishes and conditions regarding renovations.

[1638] Furthermore, the server uses an emotion engine to analyze the user's emotional data and integrates this result with a generating AI model to create more personalized renovation ideas. Based on the analyzed emotional data, the server makes suggestions that take into account the user's expectations and anxieties.

[1639] Based on the generated renovation ideas, the server uses an image generation model to create design images and videos of the finished product. This allows users to have a concrete visual image of the project. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream.

[1640] terminal

[1641] The terminal provides users with a user interface for entering detailed information about sparsely populated buildings. Through this interface, users can input the building's location, current photos, and their wishes and requirements for renovation. Furthermore, they can also input their own emotions (e.g., excitement, anticipation, anxiety, etc.).

[1642] The information entered by the user is formatted into the appropriate format by the terminal and sent to the server. The terminal also performs error checking on the information to ensure that the data is transmitted accurately.

[1643] Renovation ideas and promotional materials (images and videos) returned from the server are displayed on the device. This allows users to create concrete renovation plans based on these materials.

[1644] User

[1645] Users input detailed information and sentimental data about sparsely populated buildings they are interested in via their devices. For example, they input the building's location, size, and current use, as well as their wishes, conditions, and feelings regarding renovations (e.g., excitement, anticipation, anxiety).

[1646] Users can view renovation ideas and promotional materials displayed on their devices and use them to develop concrete project plans. The generated materials can also be used on social media and in presentations, serving as valuable tools to attract the attention of local communities and stakeholders.

[1647] Specific example

[1648] For example, consider a scenario where this system is used to generate renovation ideas for a closed elementary school located in a sparsely populated area. The user inputs the school's location, current condition (with photos), desired use (e.g., to use it as a community center), and their own feelings (e.g., expectations or anxieties about the project) from their device.

[1649] The server receives this information and generates renovation ideas using a generative AI model. The emotion engine analyzes the user's emotional data and incorporates the results into the generative AI model, providing personalized ideas that take the user's emotions into consideration.

[1650] Example of a prompt

[1651] User input:

[1652] Building details: Abandoned school in Nagano Prefecture, area 1200m² 2 Currently used as a warehouse.

[1653] Renovation request: I would like to use it as a community center.

[1654] Emotions: I'm very excited, but also a little anxious.

[1655] Expected output:

[1656] Generated from AI model: "By renovating the abandoned school into a community center, local interaction can be deepened and it can be used as a new tourist attraction. In particular, it is important to incorporate a design that prioritizes comfort. Since users have given positive feedback, let's give maximum consideration to comfort and functionality."

[1657] In this way, the present invention supports the generation and promotion of renovation ideas for buildings in sparsely populated areas, thereby contributing to regional revitalization. By introducing an emotion engine, it becomes possible to analyze user emotion data and provide more personalized suggestions.

[1658] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1659] Step 1:

[1660] The server collects local population data, traffic data, nearby facilities data, and pedestrian flow data from external APIs and databases. The collected data undergoes preprocessing such as imputing missing values, removing outliers, and standardization. For example, population data is obtained from a census database, and traffic data is obtained from a traffic management system. The input is raw data obtained from APIs and databases, and the output is preprocessed, formatted data.

[1661] Step 2:

[1662] The server uses preprocessed data to train a generative AI model using a deep learning framework (e.g., TensorFlow or PyTorch). This allows the server to build a generative AI model for generating optimal utilization ideas for sparsely populated buildings. The input is preprocessed data, and the output is the trained generative AI model. Specifically, the model is trained using training data, and its performance is evaluated.

[1663] Step 3:

[1664] Users input detailed information about sparsely populated buildings and their own feelings using a user interface on their device. This might include the building's location, current photos, renovation hopes and conditions, and emotions (e.g., excitement, anticipation, anxiety). The input is information provided by the user, and the output is formatted detailed data.

[1665] Step 4:

[1666] The terminal formats the user-entered details into the appropriate format and sends them to the server. It also performs error checking to ensure the data is sent accurately to the server. The input is the information entered by the user, and the output is the formatted data after error checking.

[1667] Step 5:

[1668] The server receives detailed information sent by the user and inputs this information into the generating AI model. Specifically, it supplies the AI ​​model with information such as the building's location, usage, and the user's wishes and conditions. The input is the detailed information received from the user, and the output is the generated renovation ideas.

[1669] Step 6:

[1670] The server analyzes emotional data and works with a generative AI model to optimize renovation ideas based on the user's emotions. The emotion engine analyzes user input, facial expressions, voice, etc., and provides emotion-based data. The input is the user's emotional data, and the output is an optimized renovation idea based on those emotions.

[1671] Step 7:

[1672] The server uses an image generation model to generate images and videos of the completed design based on the generated renovation ideas. Tools used include, for example, OpenAI's DALL-E and Google's Deep Dream. The input is the generated renovation idea, and the output is the images and videos of the completed design.

[1673] Step 8:

[1674] The server returns the generated renovation ideas and design images to the terminal. This allows the user to have a concrete visual image. The input is the generated renovation ideas and design images, and the output is the data retu...

Claims

1. Means for collecting local population data, traffic data, nearby facility data, and pedestrian flow data, Means for preprocessing the collected data, A means for training a generative AI model that generates optimal utilization ideas for sparsely populated buildings using preprocessed data, A means of receiving detailed information about sparsely populated buildings from users, A means of inputting received detailed information into a generating AI model to generate renovation ideas, A means for generating a completed design image using an image generation model based on the generated idea, A means of returning the generated renovation ideas and design images to the user, A system that includes this.

2. The system according to claim 1, further comprising means for providing the generated renovation ideas and design images as promotional materials.

3. The system according to claim 1, further comprising means for providing a user interface and enabling a user to input detailed information about sparsely populated buildings.

Images