system

Abstract

We provide the system. [Solution] A means for collecting user attribute information and generating design proposals based on a generative model, A means of projecting the generated design proposal as a three-dimensional model into a virtual space, A system that includes means for revising design proposals based on user feedback.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When considering house design, a problem faced by many users is that it is difficult to specifically show a design plan that suits their individual lifestyles and family compositions. Furthermore, the trial and error of design is complex and time-consuming, and visualization is difficult, resulting in many cases of regret after completion. The present invention aims to solve these problems and enable users to easily design their ideal living spaces.

Means for Solving the Problems

[0005] This invention is a system that automatically generates design proposals using a generative model based on attribute information collected from users. The generated design proposals are projected into a virtual space as three-dimensional models, which users can visually confirm. Furthermore, by receiving user feedback in real time and immediately reflecting it in the design proposals, the system makes it easy to realize ideal housing designs. In addition, by making modifications to adapt to market trends, regional characteristics, laws and regulations, and environmental conditions, it enables realistic and highly practical design proposals.

[0006] A "user" is someone who utilizes the system and provides information about their preferences and attributes regarding residential design.

[0007] "Attribute information" refers to data on personal or family characteristics, including a user's lifestyle, family structure, hobbies, and future plans.

[0008] A "generative model" is an algorithm or program that analyzes data collected from users and automatically generates housing design proposals.

[0009] A "design proposal" is a proposed or planned house design based on the user's requirements and conditions.

[0010] A "three-dimensional model" is a digital model that represents a residential design plan in three dimensions and is used for visualization.

[0011] A "virtual space" is a digital environment constructed using computer graphics technology, where users can visually and experientially tour a house.

[0012] "Feedback" refers to information that users provide regarding design proposals, including opinions and requests for revisions.

[0013] "Market trends" refer to information that indicates current trends and user preferences that influence housing design.

[0014] "Regional characteristics" refers to information indicating the customs, culture, and geographical conditions specific to the location where the design proposal is actually applied.

[0015] "Laws and regulations" refer to the laws and regulations that should be complied with regarding housing design.

[0016] "Environmental conditions" refer to the natural environment and factors related to sustainability that affect housing design.

Brief Explanation of Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention provides a system that allows users to easily realize their ideal home design based on their lifestyle, family structure, hobbies, and future life plans, without requiring any specialized knowledge.

[0039] The server first receives attribute information provided by the user. This information includes detailed data that reflects the user's housing needs, such as lifestyle, preferences, and future plans. For example, suppose the server is given information that the user enjoys gardening and desires a large garden.

[0040] Subsequently, the server uses a generative model to analyze this attribute information. This analysis generates an initial house design proposal that matches the user's needs. The generated design proposal includes floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0041] The terminal receives the design proposal generated from the server and converts it into a three-dimensional model. Next, the terminal projects this three-dimensional model into a virtual space so that the user can experience a virtual tour. This allows the user to take a realistic tour as if they were actually walking through the house.

[0042] During the virtual tour, users can provide feedback on areas where the design needs adjustment. For example, they might want a larger kitchen or change the location of a window. User feedback is sent to the server in real time, and the server takes this feedback into consideration when revising the design.

[0043] The server uses AI agents to make adjustments to the design, taking into account market trends, regional characteristics, regulations, and environmental conditions. This process is repeated until the house design is completed in a way that satisfies the user. This system allows users to easily materialize their ideal living space and create a satisfying design.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] Users enter detailed information about their lifestyle, family structure, hobbies, and future life plans using a web form or a dedicated application. This information is stored on the device as attribute information necessary for housing design.

[0047] Step 2:

[0048] The terminal sends the collected attribute information to the server. This data is used as the basis for generating the design plan.

[0049] Step 3:

[0050] The server analyzes the received attribute information. Using a generative model, it automatically generates an initial house design proposal that best suits the user's needs. This design proposal includes floor plans and spatial arrangements that meet the user's specified requirements.

[0051] Step 4:

[0052] The server creates a three-dimensional model of the generated design data. This model is a detailed digital representation used later for virtual tours.

[0053] Step 5:

[0054] The device sets up the VR environment necessary for the user to conduct a virtual tour and projects a three-dimensional model into the virtual space, allowing the user to experience the interior of the house.

[0055] Step 6:

[0056] Users take a virtual tour of a house using a VR headset or compatible device, identifying aspects of the design and structure they dislike or would like to modify. They input this feedback into their device via voice or an interface during the virtual tour.

[0057] Step 7:

[0058] The device sends user feedback to the server. The server analyzes the received feedback and instructs the AI ​​agent to revise the design proposal.

[0059] Step 8:

[0060] The server remodels the design proposal, incorporating the feedback, into a 3D model and sends the updated model to the terminal. This process is repeated until the user is satisfied.

[0061] The above is the basic processing flow of the system, and through this series of steps, users can design a house that is as close to their ideal as possible.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] Traditionally, the residential design process has often required users to possess specialized knowledge, resulting in significant time and expense in realizing their ideal home. Furthermore, expert advice was essential for designing homes that take market trends and regional characteristics into account, which also represented a high hurdle for users. This invention solves these problems and provides a system that allows users to realize their ideal residential design without requiring specialized knowledge.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for collecting user attribute information through an input device and transmitting it as digital data, means for generating an initial design proposal using a generative AI model to analyze the user attribute information, and means for having a display device for converting the generated design proposal into a three-dimensional model and projecting it into a virtual space. This makes it possible for users to materialize a housing design that meets their needs while making real-time adjustments, even without specialized knowledge.

[0067] A "user" refers to an individual or organization that uses this system to provide their attribute information and engage in residential design.

[0068] "Attribute information" refers to information necessary for housing design, such as the user's lifestyle, family structure, hobbies, and future life plans.

[0069] A "generative AI model" is an artificial intelligence technology used to analyze user attribute information and generate optimal housing design proposals.

[0070] "Initial design proposal" refers to the prototype of a house design that is first created by a generative AI model based on the user's attribute information.

[0071] A "three-dimensional model" refers to a model created on a computer in a virtual three-dimensional space, allowing users to visually confirm design proposals.

[0072] A "virtual space" refers to a virtual environment created through computer simulation that is different from the real world.

[0073] "Feedback" refers to input information such as opinions, requests for adjustments, and suggestions for improvement that users provide regarding design proposals.

[0074] "Information processing means" refers to a combination of software and hardware used to analyze received digital data and make revisions to design proposals or new suggestions.

[0075] "Market trend information" refers to materials and data that show recent trends and user demand necessary for residential design.

[0076] "Regional characteristics information" refers to information that indicates the unique climate, culture, and building standards of a particular geographical area.

[0077] This system operates with a three-party relationship: user, server, and terminal. First, the user inputs attribute information such as their lifestyle, family structure, hobbies, and future life plans through the terminal's input device. This information is formatted as digital data and sent to the server. A concrete example of input would be something like, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0078] The server analyzes data using a generative AI model based on attribute information received from the user. This AI model plays a crucial role in creating initial housing design proposals that match the user's needs. The generated design proposals include floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0079] Subsequently, the terminal receives the design proposal sent from the server and uses existing 3D engines (e.g., Unity or Unreal Engine) as software to convert it into a three-dimensional model. This allows the terminal to project the design proposal into a virtual space, providing the user with a realistic virtual tour experience. Through this virtual tour, the user can experience what it's like to actually walk through the house. A concrete example of a prompt might be, "An eco-friendly house with a living room that comfortably accommodates a family of four."

[0080] During the virtual tour, users can provide feedback on the design. This feedback is sent to the server in real time via the terminal. For example, a user might input requests such as "I want a larger kitchen" or "I want more light in the living room." The server receives this feedback and uses the generated AI model and information processing device to revise the design proposal. This process, which considers market trends, regional characteristics, laws, and environmental conditions, is repeated to produce the optimal proposal, resulting in a house design that matches the user's ideal.

[0081] This system makes it easy for users to design a home that meets their needs, even without specialized knowledge.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] Users use the input device on their terminal to enter attribute information such as their lifestyle, family structure, hobbies, and future life plans. The entered data is formatted as digital data and sent to the server. At this stage, users provide specific information such as, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0085] Step 2:

[0086] The server analyzes the digital data received from the user. A generative AI model is used for this analysis, and an initial design proposal is generated based on the user's attribute information. Based on the input attribute information, the generative AI model proposes floor plans and spatial arrangements, taking into account relevant design patterns and trends. As a result, a "3-bedroom design with a large garden and gardening space" is generated as the initial design proposal.

[0087] Step 3:

[0088] The server sends the generated initial design proposal to the terminal. The terminal then generates a 3D model based on this design proposal. Using a 3D engine (e.g., Unity or Unreal Engine), the design proposal is converted into a visual representation in a virtual space. The output is a 3D model that the user can visually recognize, and the virtual tour is ready.

[0089] Step 4:

[0090] The terminal uses the generated 3D model to provide users with a virtual tour. Users use display devices such as VR headsets to realistically experience the digitized interior of the house. Users can actually walk through and examine a "living room where a family of four can comfortably spend time," and get a feel for the design.

[0091] Step 5:

[0092] Users provide specific feedback during the virtual tour. For example, they can input requests such as "make the kitchen larger" via voice or text into their device. User requests are collected and sent to the server as feedback.

[0093] Step 6:

[0094] The server analyzes feedback received from the terminal and modifies the design proposal. It utilizes generative AI models and information processing tools to optimize the design proposal, taking into account the needs of new users. It updates the design by referring to databases of market trends, regional characteristics, laws, and environmental conditions. This process is repeated until the user is satisfied.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] It is difficult for users without specialized knowledge to realize their ideal home design that matches their lifestyle, family structure, hobbies, and future plans. Furthermore, there is a lack of timely means to reflect specific user feedback on design proposals and for users to easily and intuitively check the quality of the design using virtual tours. The challenge lies in quickly and flexibly revising design proposals to meet user needs.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space; means for modifying the design proposals based on user feedback; means for the user to move around within the three-dimensional model during a virtual tour and grasp the details of specific parts; and means for collecting real-time feedback during movement and immediately modifying the design proposals. This enables users to realize an ideal house design that immediately reflects their needs and to provide feedback while experiencing the design process with intuitive operation.

[0100] "Attribute information" refers to data that indicates an individual's preferences and current situation, such as a user's lifestyle, family structure, hobbies, and future plans.

[0101] A "generative model" is an algorithm or program that uses a computer to generate housing design proposals based on a user's attribute information.

[0102] A "design proposal" refers to the initial design plan or floor plan of a house generated based on the user's needs.

[0103] A "three-dimensional model" is a digital representation of a virtual space expressed within a computer-generated three-dimensional space.

[0104] "Virtual tour" is a process in which users explore a three-dimensional model in a virtual space through a digital device and visually confirm the design.

[0105] "Feedback" refers to the input of opinions and requests for revisions that users provide regarding the design proposal, and this information is reflected in the design by the system.

[0106] "Real-time" refers to a time concept where responses and processing occur immediately in response to user input or actions.

[0107] A description of the embodiment for carrying out the invention will be provided.

[0108] In this system, the user, the server, and the terminal play three main roles.

[0109] Users enter their personal attribute information into the application. This includes information about their lifestyle, family structure, hobbies, and future plans, and this information forms the basis of the system.

[0110] The server uses the received attribute information to generate an initial house design proposal tailored to the user, utilizing a generative AI model. This generative AI model can employ advanced algorithms such as GPT-3®. The generated design proposal includes floor plans and digital designs based on the user's needs.

[0111] Subsequently, the terminal receives the design proposal from the server and projects it into the virtual space as a three-dimensional model. Suitable platforms for this are three-dimensional modeling engines such as Unity and Unreal Engine. The terminal utilizes devices such as smartphones or head-mounted displays (e.g., Oculus Quest) to provide the user with a virtual tour experience.

[0112] During the virtual tour, users can freely move around the virtual space, examine details, and provide feedback. For example, they can send feedback such as "I want a bigger kitchen" via voice input or touch controls, and this information is immediately sent to the server and incorporated into the design.

[0113] The server processes feedback in real time, modifies the design proposal, and then sends the updated 3D model back to the terminal. This process allows users to intuitively and efficiently realize their ideal home design.

[0114] For example, if a user enters a prompt message such as "I want to make the living room window a little bigger," the server will immediately modify the design proposal based on that request and provide the user with an updated 3D model.

[0115] Examples of prompt statements are as follows:

[0116] User: I'd like to make the living room window a little bigger.

[0117] System: We have received user feedback. We are adjusting the window design.

[0118] User: I want to expand my kitchen space.

[0119] System: A new design proposal has been displayed. Please see this kitchen space.

[0120] This allows users to quickly and flexibly experience home designs that reflect their needs and intuitively make the necessary changes.

[0121] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0122] Step 1:

[0123] Users input attribute information such as lifestyle, family structure, hobbies, and future plans into a terminal. This input information is sent from the terminal to the server. This input data forms the basis for subsequent analysis by a generative model.

[0124] Step 2:

[0125] The server performs analysis using a generated AI model (e.g., GPT-3) based on the received user attribute information. This analysis extracts design requirements that meet the user's needs and generates an initial design proposal for a house. The output design proposal includes digital data such as floor plans and spatial designs.

[0126] Step 3:

[0127] The server sends the generated design proposal to the terminal. The terminal converts the received design proposal into a three-dimensional model. This conversion uses a three-dimensional modeling engine such as Unity or Unreal Engine. The output is a three-dimensional model in a virtual space that the user can visually confirm.

[0128] Step 4:

[0129] Users conduct virtual tours using their devices. They can freely move around within the 3D model and provide feedback on specific designs through the interface. For example, users can input prompts such as "I want to make the kitchen bigger" using voice input or touch controls.

[0130] Step 5:

[0131] The terminal collects user feedback and sends it to the server. The server processes this feedback in real time and performs data processing to incorporate it into the design proposal. Specifically, it analyzes the feedback content and makes design changes to the areas that were pointed out.

[0132] Step 6:

[0133] The server sends the revised design proposal back to the terminal. This updated design proposal is converted into a new 3D model and displayed to the user. The user takes another virtual tour and provides further feedback if necessary.

[0134] This series of processes allows users to efficiently realize their ideal living space.

[0135] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0136] This invention incorporates an emotion engine into a system that automates home design based on a user's lifestyle, family structure, hobbies, and future plans. The emotion engine recognizes the user's emotions and optimizes the design accordingly.

[0137] The server first receives attribute information from the user. This attribute information includes the user's requests and expectations regarding the house design. The server uses a generative model to analyze the user's data and generate initial house design proposals.

[0138] Next, the server creates a 3D model of the generated design proposal and projects it into a virtual space via the terminal. The terminal then uses a VR device to provide this virtual space to the user, allowing the user to actually view the design proposal.

[0139] In addition, this system is equipped with an emotion engine. The terminal analyzes the user's facial expressions and voice to recognize their current emotional state. For example, if the emotion engine detects facial expressions indicating dissatisfaction or surprise while the user is taking a virtual tour, it sends this information to the server. Based on this emotional data, the server adjusts the design to correspond to the user's emotional state. For example, it might suggest design changes that emphasize areas where the user is feeling happy.

[0140] User feedback based on emotions is processed on the server in real time. The AI ​​agent then modifies the design proposal based on market trends, regional characteristics, laws, and environmental conditions, and presents the 3D model to the user again.

[0141] A key feature of this system is that it provides an intuitive design process that takes user emotions into consideration, enabling efficient guidance to arrive at the optimal design for the user. Through emotion-based feedback, users can achieve a more satisfying home design.

[0142] The following describes the processing flow.

[0143] Step 1:

[0144] Users input information about their lifestyle, family structure, hobbies, and future plans through their devices. This allows for the collection of user attribute information.

[0145] Step 2:

[0146] The terminal sends the collected attribute information to the server. The server then prepares to analyze this information.

[0147] Step 3:

[0148] The server uses a generative model to analyze the user's attribute information and generate an initial house design proposal. This design proposal is customized based on the user's requests.

[0149] Step 4:

[0150] The server converts the generated design proposal into a three-dimensional model. This model is later used for projection in the virtual space.

[0151] Step 5:

[0152] The terminal projects the 3D model received from the server into a virtual space. The user can use a VR device to freely move around in this space and virtually tour the design proposal.

[0153] Step 6:

[0154] During a virtual tour, the system analyzes the user's facial expressions and voice through the device's camera and microphone using an emotion engine. This allows for real-time recognition of the user's emotional state.

[0155] Step 7:

[0156] The emotion engine recognizes the user's emotions, which are then sent from the device to the server. The server uses this emotion data to revise the design proposal.

[0157] Step 8:

[0158] The server uses an AI agent to adjust design proposals based on the user's emotions. For example, if a user expresses pleasure with a particular design, it will generate a revised proposal that emphasizes that element.

[0159] Step 9:

[0160] The server generates the adjusted design proposal again as a 3D model and sends it to the terminal.

[0161] Step 10:

[0162] The device projects the updated 3D model into a virtual space, presenting the user with new design proposals. This process is repeated until the user is satisfied with the design.

[0163] In this way, design proposals that reflect the user's feelings are made and revised, enabling the user to realize the optimal housing design.

[0164] (Example 2)

[0165] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0166] Conventional housing design systems struggle to optimize designs by considering user attribute information and emotions, lacking a design process that enhances user emotional satisfaction. Furthermore, they struggle to provide designs that effectively reflect market trends, regional characteristics, laws, and environmental conditions. This creates a challenge in quickly and efficiently proposing homes that truly meet user needs.

[0167] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0168] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model, means for projecting the generated design proposals as three-dimensional models into a virtual space, and means for analyzing the user's facial expressions and voice to recognize their emotional state. This makes it possible to optimize the design according to the user's emotions and provide design proposals that take into account market trends, regional characteristics, laws and regulations, and environmental conditions.

[0169] "User attribute information" refers to information that includes individual needs and expectations, such as the user's lifestyle, family structure, hobbies, and future plans.

[0170] "Methods for generating design proposals based on generative models" refer to methods that utilize collected attribute information and automatically generate residential design proposals using artificial intelligence and machine learning technologies.

[0171] "A means of projecting a three-dimensional model into a virtual space" refers to a technology that visualizes the generated design proposal three-dimensionally as digital data and projects it into a virtual environment that users can visually confirm.

[0172] "Means for analyzing a user's facial expressions and voice to recognize their emotional state" refers to a method of analyzing a user's facial expressions and voice tone using sensors and analysis software to identify the user's emotional response.

[0173] "Optimization" refers to the process of improving design proposals based on user attribute information and emotional states, and adjusting the design to enhance user satisfaction.

[0174] "Market trends, regional characteristics, laws, and environmental conditions" refers to current market needs, unique regional characteristics, relevant laws and regulations, and environmental constraints and considerations.

[0175] This invention provides a system that automates house design based on user attribute information, and optimizes the design by taking into account the user's emotional state during the process. The system mainly consists of a server, terminals, and user interaction.

[0176] First, the server receives attribute information from the user regarding their lifestyle, family structure, hobbies, future plans, etc. This information reflects the user's specific requests and expectations for their home design. Based on the collected attribute information, the server uses a generative AI model to generate initial home design proposals. This generative AI model takes specific prompt statements as input and outputs design proposals. An example of a prompt statement would be, "Please recommend a home design for a single musician in their 30s, including an open music room that incorporates plenty of natural light."

[0177] Next, the server converts the generated design proposal into a three-dimensional model using 3D modeling software. This 3D model is represented in a virtual space designed to allow users to virtually tour the interior. The completed 3D model is sent to a terminal, and users can experience it using a VR device.

[0178] The device is equipped with a camera and microphone to analyze the user's facial expressions and voice during the virtual tour, and uses an emotion engine to recognize emotional states in real time. This process monitors the user's facial expressions and voice tone in real time and determines which emotion they correspond to, such as joy, surprise, or dissatisfaction.

[0179] The recognized emotion information is sent to the server, which optimizes the design proposal based on the user's emotional state. For example, areas where the user felt joy are highlighted or made more appealing in the design suggestions. Furthermore, an AI agent is responsible for revising the design proposal, taking into account market trends, regional characteristics, laws and regulations, and environmental conditions, and then re-transmitting the improved 3D model to the terminal.

[0180] This system allows users to provide emotional feedback, enabling more personalized designs and ultimately increasing satisfaction with the home design.

[0181] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0182] Step 1:

[0183] The server receives attribute information from the user as input. Specifically, the user enters information about their lifestyle, family structure, hobbies, and future plans. This information is stored in a database and digitized for use in subsequent processing.

[0184] Step 2:

[0185] The server inputs the received attribute information as a prompt to the generating AI model. An example of a prompt is, "Please propose a design for a 2LDK apartment including a gardening space." Based on this input, the generating AI model generates an initial residential design and outputs it as design data.

[0186] Step 3:

[0187] The server takes the generated design proposal as input and converts it into a three-dimensional model using 3D modeling software. Specifically, it renders each element of the design proposal into a three-dimensional shape and outputs it in a format suitable for the virtual space.

[0188] Step 4:

[0189] The terminal receives a 3D model sent from the server and projects it into a virtual space using a VR device. Users can directly view the design proposal and experience its interior within the VR space.

[0190] Step 5:

[0191] Users take a virtual tour through a VR device, and their facial expressions and voice are recorded in real time by the device's camera and microphone. This data is treated as input data indicating the user's emotions.

[0192] Step 6:

[0193] The device uses an emotion engine to analyze input facial and voice data and output the user's emotional state. Specifically, it uses facial recognition algorithms and voice analysis to identify emotions such as joy, surprise, and dissatisfaction.

[0194] Step 7:

[0195] The server takes the analyzed sentiment data as input and optimizes the design proposal. Based on the sentiment data, it suggests design changes, such as highlighting aspects that users found favorable, and generates the revised design data.

[0196] Step 8:

[0197] The server uses an AI agent to further refine the optimized design proposal, taking into account market trends, regional characteristics, laws, and environmental conditions. This results in the output of a final design proposal that conforms to laws and local regulations.

[0198] Step 9:

[0199] The server then sends the 3D model of the final design proposal back to the terminal, presenting the revised proposal to the user. This allows the user to review the updated design proposal and evaluate the differences.

[0200] (Application Example 2)

[0201] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0202] In residential design, providing an individually optimized living environment that considers not only residents' attribute information but also their emotional state has been difficult with conventional technologies. There is a need for methods that can significantly improve resident satisfaction by analyzing users' emotions in real time and providing experiences that align with those emotions.

[0203] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0204] In this invention, the server includes means for collecting user attribute information and emotional state and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space and modifying the design proposals based on emotional feedback; and means for analyzing the user's emotions in the virtual space in real time using emotion recognition technology and reflecting these in the design proposals. This makes it possible to provide users with design proposals optimized for their needs and to realize an experience that aligns with their emotions.

[0205] A "user" is an individual who uses this system to request a house design.

[0206] "Attribute information" refers to data such as the user's lifestyle, family structure, hobbies, and future plans.

[0207] "Emotional state" refers to information about the user's psychological response, extracted from their facial expressions, voice, and other similar data.

[0208] A "generative model" is an algorithm that generates design proposals based on user attribute information and emotional state.

[0209] A "design proposal" is an initial plan for a house that takes into account the user's needs and feelings.

[0210] A "three-dimensional model" is a three-dimensional data representation that visually shows a design proposal in a virtual space.

[0211] A "virtual space" is a digital environment created by a computer system that is different from reality.

[0212] "Emotional feedback" refers to response information based on the user's emotional state.

[0213] "Emotion recognition technology" refers to technical methods for analyzing a user's emotional state.

[0214] "Real-time" refers to a time concept that describes the process by which a user's emotional state is analyzed and reflected immediately.

[0215] The system implementing this invention generates and presents individually optimized housing designs in a virtual space based on attribute information such as the user's lifestyle, family structure, hobbies, and future plans, as well as emotional states obtained from facial expressions and voice using emotion recognition technology. The server collects the user's attribute information and generates design proposals using a generation AI model. The generated design proposals are projected as three-dimensional models into a virtual space via a VR device on the terminal. The user experiences the design proposals in this virtual space, and their emotional state is analyzed in real time using emotion recognition technology. Based on the emotional feedback, the server modifies the design proposals and provides the user with optimized designs. This makes it possible to realize an experience that aligns with emotions and achieve highly satisfying housing designs.

[0216] Specifically, the device uses an emotion recognition API to analyze the user's emotional state in real time through its camera and microphone. The server uses a generative AI model to generate new design proposals based on the collected attribute information and emotional state. The regenerated design proposals are then converted into 3D models and presented to the user in a virtual space on the device. The software used includes, for example, 3D engines such as Unity and Unreal Engine, and cloud services such as Amazon Web Services and Google Cloud Platform.

[0217] For example, if a user expresses emotions such as "surprise" or "joy" while viewing the living room design during a virtual tour, the system will use this reaction to suggest ways to further enhance the living room design. An example of a prompt might be, "If the user is enjoying the living room, suggest ways to further enhance its design."

[0218] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0219] Step 1:

[0220] The server receives attribute information from the user. As input, it collects data such as lifestyle, family structure, hobbies, and future plans. This provides the basic information needed to generate design proposals optimized for the user.

[0221] Step 2:

[0222] The server inputs the collected attribute information into a generating AI model to generate design proposals. The generating AI model extracts the necessary conditions for the design from the attribute information and outputs a design proposal based on those conditions. This creates a personalized house design proposal for the user.

[0223] Step 3:

[0224] The terminal projects the generated design proposal as a three-dimensional model into the virtual space. Using the design proposal received from the server as input, it creates a visual model that the user can actually experience using a 3D engine. This allows the user to virtually tour the design proposal.

[0225] Step 4:

[0226] Users explore a virtual space through a VR device. This allows users to review design proposals and intuitively grasp the layout and arrangement of each room. As a result, users can develop genuine opinions and feelings about the design.

[0227] Step 5:

[0228] The device uses emotion recognition technology to analyze the user's facial expressions and voice, detecting their emotional state in real time. It acquires the user's camera video and audio data as input and processes it using an emotion recognition API. This allows it to acquire and analyze current emotional data based on the user's reactions.

[0229] Step 6:

[0230] The server modifies the design proposal based on the detected emotional state. The AI ​​model receives emotional data as input and recalculates the design proposal, outputting it as an output. This ensures that the design elements that should be emphasized are reflected according to the user's emotions.

[0231] Step 7:

[0232] The server then remodels the revised design proposal into a 3D model and transfers it to the terminal. The terminal projects the new design proposal into the virtual space. This allows the user to experience the latest design proposal, which has been adjusted based on emotions, once again.

[0233] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0234] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0235] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0236] [Second Embodiment]

[0237] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0238] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0239] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0240] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0241] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0242] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0243] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0244] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0245] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0246] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0247] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0248] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0249] This invention provides a system that allows users to easily realize their ideal home design based on their lifestyle, family structure, hobbies, and future life plans, without requiring any specialized knowledge.

[0250] The server first receives attribute information provided by the user. This information includes detailed data that reflects the user's housing needs, such as lifestyle, preferences, and future plans. For example, suppose the server is given information that the user enjoys gardening and desires a large garden.

[0251] Subsequently, the server uses a generative model to analyze this attribute information. This analysis generates an initial house design proposal that matches the user's needs. The generated design proposal includes floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0252] The terminal receives the design proposal generated from the server and converts it into a three-dimensional model. Next, the terminal projects this three-dimensional model into a virtual space so that the user can experience a virtual tour. This allows the user to take a realistic tour as if they were actually walking through the house.

[0253] During the virtual tour, users can provide feedback on areas where the design needs adjustment. For example, they might want a larger kitchen or change the location of a window. User feedback is sent to the server in real time, and the server takes this feedback into consideration when revising the design.

[0254] The server uses AI agents to make adjustments to the design, taking into account market trends, regional characteristics, regulations, and environmental conditions. This process is repeated until the house design is completed in a way that satisfies the user. This system allows users to easily materialize their ideal living space and create a satisfying design.

[0255] The following describes the processing flow.

[0256] Step 1:

[0257] Users enter detailed information about their lifestyle, family structure, hobbies, and future life plans using a web form or a dedicated application. This information is stored on the device as attribute information necessary for housing design.

[0258] Step 2:

[0259] The terminal sends the collected attribute information to the server. This data is used as the basis for generating the design plan.

[0260] Step 3:

[0261] The server analyzes the received attribute information. Using a generative model, it automatically generates an initial house design proposal that best suits the user's needs. This design proposal includes floor plans and spatial arrangements that meet the user's specified requirements.

[0262] Step 4:

[0263] The server creates a three-dimensional model of the generated design data. This model is a detailed digital representation used later for virtual tours.

[0264] Step 5:

[0265] The device sets up the VR environment necessary for the user to conduct a virtual tour and projects a three-dimensional model into the virtual space, allowing the user to experience the interior of the house.

[0266] Step 6:

[0267] Users take a virtual tour of a house using a VR headset or compatible device, identifying aspects of the design and structure they dislike or would like to modify. They input this feedback into their device via voice or an interface during the virtual tour.

[0268] Step 7:

[0269] The device sends user feedback to the server. The server analyzes the received feedback and instructs the AI ​​agent to revise the design proposal.

[0270] Step 8:

[0271] The server remodels the design proposal, incorporating the feedback, into a 3D model and sends the updated model to the terminal. This process is repeated until the user is satisfied.

[0272] The above is the basic processing flow of the system, and through this series of steps, users can design a house that is as close to their ideal as possible.

[0273] (Example 1)

[0274] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0275] Traditionally, the residential design process has often required users to possess specialized knowledge, resulting in significant time and expense in realizing their ideal home. Furthermore, expert advice was essential for designing homes that take market trends and regional characteristics into account, which also represented a high hurdle for users. This invention solves these problems and provides a system that allows users to realize their ideal residential design without requiring specialized knowledge.

[0276] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0277] In this invention, the server includes means for collecting user attribute information through an input device and transmitting it as digital data, means for generating an initial design proposal using a generative AI model to analyze the user attribute information, and means for having a display device for converting the generated design proposal into a three-dimensional model and projecting it into a virtual space. This makes it possible for users to materialize a housing design that meets their needs while making real-time adjustments, even without specialized knowledge.

[0278] A "user" refers to an individual or organization that uses this system to provide their attribute information and engage in residential design.

[0279] "Attribute information" refers to information necessary for housing design, such as the user's lifestyle, family composition, hobbies, future life plans, etc.

[0280] "Generative AI model" refers to artificial intelligence technology used to analyze the user's attribute information and generate an optimal housing design plan.

[0281] "Initial design plan" refers to the prototype of the housing design created for the first time based on the user's attribute information by the generative AI model.

[0282] "3D model" means a model in a virtual 3D space generated on a computer, where the user can visually confirm the design plan.

[0283] "Virtual space" refers to a virtual environment different from the real world created by computer simulation.

[0284] "Feedback" refers to input information such as the user's opinions, adjustment requests, and improvement proposals for the design plan.

[0285] "Information processing means" refers to a combination of software and hardware that analyzes the received digital data and makes corrections or new proposals for the design plan.

[0286] "Market trend information" refers to materials and data indicating recent trends and user demands necessary for housing design.

[0287] "Regional characteristic information" refers to information indicating the local conditions, culture, building standards, etc. unique to a specific geographical region.

[0288] This system operates with a three-party relationship: user, server, and terminal. First, the user inputs attribute information such as their lifestyle, family structure, hobbies, and future life plans through the terminal's input device. This information is formatted as digital data and sent to the server. A concrete example of input would be something like, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0289] The server analyzes data using a generative AI model based on attribute information received from the user. This AI model plays a crucial role in creating initial housing design proposals that match the user's needs. The generated design proposals include floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0290] Subsequently, the terminal receives the design proposal sent from the server and uses existing 3D engines (e.g., Unity or Unreal Engine) as software to convert it into a three-dimensional model. This allows the terminal to project the design proposal into a virtual space, providing the user with a realistic virtual tour experience. Through this virtual tour, the user can experience what it's like to actually walk through the house. A concrete example of a prompt might be, "An eco-friendly house with a living room that comfortably accommodates a family of four."

[0291] During the virtual tour, users can provide feedback on the design. This feedback is sent to the server in real time via the terminal. For example, a user might input requests such as "I want a larger kitchen" or "I want more light in the living room." The server receives this feedback and uses the generated AI model and information processing device to revise the design proposal. This process, which considers market trends, regional characteristics, laws, and environmental conditions, is repeated to produce the optimal proposal, resulting in a house design that matches the user's ideal.

[0292] This system makes it easy for users to design a home that meets their needs, even without specialized knowledge.

[0293] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0294] Step 1:

[0295] Users use the input device on their terminal to enter attribute information such as their lifestyle, family structure, hobbies, and future life plans. The entered data is formatted as digital data and sent to the server. At this stage, users provide specific information such as, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0296] Step 2:

[0297] The server analyzes the digital data received from the user. A generative AI model is used for this analysis, and an initial design proposal is generated based on the user's attribute information. Based on the input attribute information, the generative AI model proposes floor plans and spatial arrangements, taking into account relevant design patterns and trends. As a result, a "3-bedroom design with a large garden and gardening space" is generated as the initial design proposal.

[0298] Step 3:

[0299] The server sends the generated initial design proposal to the terminal. The terminal then generates a 3D model based on this design proposal. Using a 3D engine (e.g., Unity or Unreal Engine), the design proposal is converted into a visual representation in a virtual space. The output is a 3D model that the user can visually recognize, and the virtual tour is ready.

[0300] Step 4:

[0301] The terminal provides the user with a virtual tour using the generated three-dimensional model. The user uses a display device such as a VR headset to realistically experience the digitized interior of the house. The user can actually walk through and check a "living room where four family members can comfortably live" and grasp the feel of the design.

[0302] Step 5:

[0303] The user gives specific feedback during the virtual tour. For example, the user inputs a wish such as "make the kitchen larger" into the terminal by voice or text. The user's wishes are immediately collected and sent to the server as feedback.

[0304] Step 6:

[0305] The server analyzes the feedback received from the terminal and modifies the design plan. Utilizing the generated AI model and information processing means, it optimizes the design plan considering the needs of new users. It refers to a database of market trends, regional characteristics, laws, and environmental conditions to update the design. This process is repeated until the user is satisfied.

[0306] (Application Example 1)

[0307] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0308] It is difficult for a user without specialized knowledge to realize an ideal house design that suits their lifestyle, family composition, hobbies, and future plans. Also, there is a lack of means to promptly reflect the user's specific feedback on the design plan and allow the user to easily and intuitively check the quality of the design using a virtual tour. The prompt and flexible modification of the design plan according to the user's needs is an issue.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0310] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space; means for modifying the design proposals based on user feedback; means for the user to move around within the three-dimensional model during a virtual tour and grasp the details of specific parts; and means for collecting real-time feedback during movement and immediately modifying the design proposals. This enables users to realize an ideal house design that immediately reflects their needs and to provide feedback while experiencing the design process with intuitive operation.

[0311] "Attribute information" refers to data that indicates an individual's preferences and current situation, such as a user's lifestyle, family structure, hobbies, and future plans.

[0312] A "generative model" is an algorithm or program that uses a computer to generate housing design proposals based on a user's attribute information.

[0313] A "design proposal" refers to the initial design plan or floor plan of a house generated based on the user's needs.

[0314] A "three-dimensional model" is a digital representation of a virtual space expressed within a computer-generated three-dimensional space.

[0315] "Virtual tour" is a process in which users explore a three-dimensional model in a virtual space through a digital device and visually confirm the design.

[0316] "Feedback" refers to the input of opinions and requests for revisions that users provide regarding the design proposal, and this information is reflected in the design by the system.

[0317] "Real-time" refers to a time concept where responses and processing occur immediately in response to user input or actions.

[0318] A description of the embodiment for carrying out the invention will be provided.

[0319] In this system, the user, the server, and the terminal play three main roles.

[0320] Users enter their personal attribute information into the application. This includes information about their lifestyle, family structure, hobbies, and future plans, and this information forms the basis of the system.

[0321] The server uses the received attribute information to generate an initial house design proposal tailored to the user, utilizing a generative AI model. This generative AI model can employ advanced algorithms such as GPT-3. The generated design proposal includes floor plans and digital designs based on the user's needs.

[0322] Subsequently, the terminal receives the design proposal from the server and projects it into the virtual space as a three-dimensional model. Suitable platforms for this are three-dimensional modeling engines such as Unity and Unreal Engine. The terminal utilizes devices such as smartphones or head-mounted displays (e.g., Oculus Quest) to provide the user with a virtual tour experience.

[0323] During the virtual tour, users can freely move around the virtual space, examine details, and provide feedback. For example, they can send feedback such as "I want a bigger kitchen" via voice input or touch controls, and this information is immediately sent to the server and incorporated into the design.

[0324] The server processes feedback in real time, modifies the design proposal, and then sends the updated 3D model back to the terminal. This process allows users to intuitively and efficiently realize their ideal home design.

[0325] For example, if a user enters a prompt message such as "I want to make the living room window a little bigger," the server will immediately modify the design proposal based on that request and provide the user with an updated 3D model.

[0326] Examples of prompt statements are as follows:

[0327] User: I'd like to make the living room window a little bigger.

[0328] System: We have received user feedback. We are adjusting the window design.

[0329] User: I want to expand my kitchen space.

[0330] System: A new design proposal has been displayed. Please see this kitchen space.

[0331] This allows users to quickly and flexibly experience home designs that reflect their needs and intuitively make the necessary changes.

[0332] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0333] Step 1:

[0334] Users input attribute information such as lifestyle, family structure, hobbies, and future plans into a terminal. This input information is sent from the terminal to the server. This input data forms the basis for subsequent analysis by a generative model.

[0335] Step 2:

[0336] The server performs analysis using a generated AI model (e.g., GPT-3) based on the received user attribute information. This analysis extracts design requirements that meet the user's needs and generates an initial design proposal for a house. The output design proposal includes digital data such as floor plans and spatial designs.

[0337] Step 3:

[0338] The server sends the generated design proposal to the terminal. The terminal converts the received design proposal into a three-dimensional model. This conversion uses a three-dimensional modeling engine such as Unity or Unreal Engine. The output is a three-dimensional model in a virtual space that the user can visually confirm.

[0339] Step 4:

[0340] Users conduct virtual tours using their devices. They can freely move around within the 3D model and provide feedback on specific designs through the interface. For example, users can input prompts such as "I want to make the kitchen bigger" using voice input or touch controls.

[0341] Step 5:

[0342] The terminal collects user feedback and sends it to the server. The server processes this feedback in real time and performs data processing to incorporate it into the design proposal. Specifically, it analyzes the feedback content and makes design changes to the areas that were pointed out.

[0343] Step 6:

[0344] The server sends the revised design proposal back to the terminal. This updated design proposal is converted into a new 3D model and displayed to the user. The user takes another virtual tour and provides further feedback if necessary.

[0345] This series of processes allows users to efficiently realize their ideal living space.

[0346] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0347] This invention incorporates an emotion engine into a system that automates home design based on a user's lifestyle, family structure, hobbies, and future plans. The emotion engine recognizes the user's emotions and optimizes the design accordingly.

[0348] The server first receives attribute information from the user. This attribute information includes the user's requests and expectations regarding the house design. The server uses a generative model to analyze the user's data and generate initial house design proposals.

[0349] Next, the server creates a 3D model of the generated design proposal and projects it into a virtual space via the terminal. The terminal then uses a VR device to provide this virtual space to the user, allowing the user to actually view the design proposal.

[0350] In addition, this system is equipped with an emotion engine. The terminal analyzes the user's facial expressions and voice to recognize their current emotional state. For example, if the emotion engine detects facial expressions indicating dissatisfaction or surprise while the user is taking a virtual tour, it sends this information to the server. Based on this emotional data, the server adjusts the design to correspond to the user's emotional state. For example, it might suggest design changes that emphasize areas where the user is feeling happy.

[0351] User feedback based on emotions is processed on the server in real time. The AI ​​agent then modifies the design proposal based on market trends, regional characteristics, laws, and environmental conditions, and presents the 3D model to the user again.

[0352] A key feature of this system is that it provides an intuitive design process that takes user emotions into consideration, enabling efficient guidance to arrive at the optimal design for the user. Through emotion-based feedback, users can achieve a more satisfying home design.

[0353] The following describes the processing flow.

[0354] Step 1:

[0355] Users input information about their lifestyle, family structure, hobbies, and future plans through their devices. This allows for the collection of user attribute information.

[0356] Step 2:

[0357] The terminal sends the collected attribute information to the server. The server then prepares to analyze this information.

[0358] Step 3:

[0359] The server uses a generative model to analyze the user's attribute information and generate an initial house design proposal. This design proposal is customized based on the user's requests.

[0360] Step 4:

[0361] The server converts the generated design proposal into a three-dimensional model. This model is later used for projection in the virtual space.

[0362] Step 5:

[0363] The terminal projects the 3D model received from the server into a virtual space. The user can use a VR device to freely move around in this space and virtually tour the design proposal.

[0364] Step 6:

[0365] During a virtual tour, the system analyzes the user's facial expressions and voice through the device's camera and microphone using an emotion engine. This allows for real-time recognition of the user's emotional state.

[0366] Step 7:

[0367] The emotion engine recognizes the user's emotions, which are then sent from the device to the server. The server uses this emotion data to revise the design proposal.

[0368] Step 8:

[0369] The server uses an AI agent to adjust design proposals based on the user's emotions. For example, if a user expresses pleasure with a particular design, it will generate a revised proposal that emphasizes that element.

[0370] Step 9:

[0371] The server generates the adjusted design proposal again as a 3D model and sends it to the terminal.

[0372] Step 10:

[0373] The device projects the updated 3D model into a virtual space, presenting the user with new design proposals. This process is repeated until the user is satisfied with the design.

[0374] In this way, design proposals that reflect the user's feelings are made and revised, enabling the user to realize the optimal housing design.

[0375] (Example 2)

[0376] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0377] Conventional housing design systems struggle to optimize designs by considering user attribute information and emotions, lacking a design process that enhances user emotional satisfaction. Furthermore, they struggle to provide designs that effectively reflect market trends, regional characteristics, laws, and environmental conditions. This creates a challenge in quickly and efficiently proposing homes that truly meet user needs.

[0378] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0379] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model, means for projecting the generated design proposals as three-dimensional models into a virtual space, and means for analyzing the user's facial expressions and voice to recognize their emotional state. This makes it possible to optimize the design according to the user's emotions and provide design proposals that take into account market trends, regional characteristics, laws and regulations, and environmental conditions.

[0380] "User attribute information" refers to information that includes individual needs and expectations, such as the user's lifestyle, family structure, hobbies, and future plans.

[0381] "Methods for generating design proposals based on generative models" refer to methods that utilize collected attribute information and automatically generate residential design proposals using artificial intelligence and machine learning technologies.

[0382] "A means of projecting a three-dimensional model into a virtual space" refers to a technology that visualizes the generated design proposal three-dimensionally as digital data and projects it into a virtual environment that users can visually confirm.

[0383] "Means for analyzing a user's facial expressions and voice to recognize their emotional state" refers to a method of analyzing a user's facial expressions and voice tone using sensors and analysis software to identify the user's emotional response.

[0384] "Optimization" refers to the process of improving design proposals based on user attribute information and emotional states, and adjusting the design to enhance user satisfaction.

[0385] "Market trends, regional characteristics, laws, and environmental conditions" refers to current market needs, unique regional characteristics, relevant laws and regulations, and environmental constraints and considerations.

[0386] This invention provides a system that automates house design based on user attribute information, and optimizes the design by taking into account the user's emotional state during the process. The system mainly consists of a server, terminals, and user interaction.

[0387] First, the server receives attribute information from the user regarding their lifestyle, family structure, hobbies, future plans, etc. This information reflects the user's specific requests and expectations for their home design. Based on the collected attribute information, the server uses a generative AI model to generate initial home design proposals. This generative AI model takes specific prompt statements as input and outputs design proposals. An example of a prompt statement would be, "Please recommend a home design for a single musician in their 30s, including an open music room that incorporates plenty of natural light."

[0388] Next, the server converts the generated design proposal into a three-dimensional model using 3D modeling software. This 3D model is represented in a virtual space designed to allow users to virtually tour the interior. The completed 3D model is sent to a terminal, and users can experience it using a VR device.

[0389] The device is equipped with a camera and microphone to analyze the user's facial expressions and voice during the virtual tour, and uses an emotion engine to recognize emotional states in real time. This process monitors the user's facial expressions and voice tone in real time and determines which emotion they correspond to, such as joy, surprise, or dissatisfaction.

[0390] The recognized emotion information is sent to the server, which optimizes the design proposal based on the user's emotional state. For example, areas where the user felt joy are highlighted or made more appealing in the design suggestions. Furthermore, an AI agent is responsible for revising the design proposal, taking into account market trends, regional characteristics, laws and regulations, and environmental conditions, and then re-transmitting the improved 3D model to the terminal.

[0391] This system allows users to provide emotional feedback, enabling more personalized designs and ultimately increasing satisfaction with the home design.

[0392] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0393] Step 1:

[0394] The server receives attribute information from the user as input. Specifically, the user enters information about their lifestyle, family structure, hobbies, and future plans. This information is stored in a database and digitized for use in subsequent processing.

[0395] Step 2:

[0396] The server inputs the received attribute information as a prompt to the generating AI model. An example of a prompt is, "Please propose a design for a 2LDK apartment including a gardening space." Based on this input, the generating AI model generates an initial residential design and outputs it as design data.

[0397] Step 3:

[0398] The server takes the generated design proposal as input and converts it into a three-dimensional model using 3D modeling software. Specifically, it renders each element of the design proposal into a three-dimensional shape and outputs it in a format suitable for the virtual space.

[0399] Step 4:

[0400] The terminal receives a 3D model sent from the server and projects it into a virtual space using a VR device. Users can directly view the design proposal and experience its interior within the VR space.

[0401] Step 5:

[0402] Users take a virtual tour through a VR device, and their facial expressions and voice are recorded in real time by the device's camera and microphone. This data is treated as input data indicating the user's emotions.

[0403] Step 6:

[0404] The device uses an emotion engine to analyze input facial and voice data and output the user's emotional state. Specifically, it uses facial recognition algorithms and voice analysis to identify emotions such as joy, surprise, and dissatisfaction.

[0405] Step 7:

[0406] The server takes the analyzed sentiment data as input and optimizes the design proposal. Based on the sentiment data, it suggests design changes, such as highlighting aspects that users found favorable, and generates the revised design data.

[0407] Step 8:

[0408] The server uses an AI agent to further refine the optimized design proposal, taking into account market trends, regional characteristics, laws, and environmental conditions. This results in the output of a final design proposal that conforms to laws and local regulations.

[0409] Step 9:

[0410] The server then sends the 3D model of the final design proposal back to the terminal, presenting the revised proposal to the user. This allows the user to review the updated design proposal and evaluate the differences.

[0411] (Application Example 2)

[0412] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0413] In residential design, providing an individually optimized living environment that considers not only residents' attribute information but also their emotional state has been difficult with conventional technologies. There is a need for methods that can significantly improve resident satisfaction by analyzing users' emotions in real time and providing experiences that align with those emotions.

[0414] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0415] In this invention, the server includes means for collecting user attribute information and emotional state and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space and modifying the design proposals based on emotional feedback; and means for analyzing the user's emotions in the virtual space in real time using emotion recognition technology and reflecting these in the design proposals. This makes it possible to provide users with design proposals optimized for their needs and to realize an experience that aligns with their emotions.

[0416] A "user" is an individual who uses this system to request a house design.

[0417] "Attribute information" refers to data such as the user's lifestyle, family structure, hobbies, and future plans.

[0418] "Emotional state" refers to information about the user's psychological response, extracted from their facial expressions, voice, and other similar data.

[0419] A "generative model" is an algorithm that generates design proposals based on user attribute information and emotional state.

[0420] A "design proposal" is an initial plan for a house that takes into account the user's needs and feelings.

[0421] A "three-dimensional model" is a three-dimensional data representation that visually shows a design proposal in a virtual space.

[0422] A "virtual space" is a digital environment created by a computer system that is different from reality.

[0423] "Emotional feedback" refers to response information based on the user's emotional state.

[0424] "Emotion recognition technology" refers to technical methods for analyzing a user's emotional state.

[0425] "Real-time" refers to a time concept that describes the process by which a user's emotional state is analyzed and reflected immediately.

[0426] The system implementing this invention generates and presents individually optimized housing designs in a virtual space based on attribute information such as the user's lifestyle, family structure, hobbies, and future plans, as well as emotional states obtained from facial expressions and voice using emotion recognition technology. The server collects the user's attribute information and generates design proposals using a generation AI model. The generated design proposals are projected as three-dimensional models into a virtual space via a VR device on the terminal. The user experiences the design proposals in this virtual space, and their emotional state is analyzed in real time using emotion recognition technology. Based on the emotional feedback, the server modifies the design proposals and provides the user with optimized designs. This makes it possible to realize an experience that aligns with emotions and achieve highly satisfying housing designs.

[0427] Specifically, the device uses an emotion recognition API to analyze the user's emotional state in real time through its camera and microphone. The server uses a generative AI model to generate new design proposals based on the collected attribute information and emotional state. The regenerated design proposals are then converted into 3D models and presented to the user in a virtual space on the device. The software used includes, for example, 3D engines such as Unity and Unreal Engine, and cloud services such as Amazon Web Services and Google Cloud Platform.

[0428] For example, if a user expresses emotions such as "surprise" or "joy" while viewing the living room design during a virtual tour, the system will use this reaction to suggest ways to further enhance the living room design. An example of a prompt might be, "If the user is enjoying the living room, suggest ways to further enhance its design."

[0429] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0430] Step 1:

[0431] The server receives attribute information from the user. As input, it collects data such as lifestyle, family structure, hobbies, and future plans. This provides the basic information needed to generate design proposals optimized for the user.

[0432] Step 2:

[0433] The server inputs the collected attribute information into a generating AI model to generate design proposals. The generating AI model extracts the necessary conditions for the design from the attribute information and outputs a design proposal based on those conditions. This creates a personalized house design proposal for the user.

[0434] Step 3:

[0435] The terminal projects the generated design proposal as a three-dimensional model into the virtual space. Using the design proposal received from the server as input, it creates a visual model that the user can actually experience using a 3D engine. This allows the user to virtually tour the design proposal.

[0436] Step 4:

[0437] Users explore a virtual space through a VR device. This allows users to review design proposals and intuitively grasp the layout and arrangement of each room. As a result, users can develop genuine opinions and feelings about the design.

[0438] Step 5:

[0439] The device uses emotion recognition technology to analyze the user's facial expressions and voice, detecting their emotional state in real time. It acquires the user's camera video and audio data as input and processes it using an emotion recognition API. This allows it to acquire and analyze current emotional data based on the user's reactions.

[0440] Step 6:

[0441] The server modifies the design proposal based on the detected emotional state. The AI ​​model receives emotional data as input and recalculates the design proposal, outputting it as an output. This ensures that the design elements that should be emphasized are reflected according to the user's emotions.

[0442] Step 7:

[0443] The server then remodels the revised design proposal into a 3D model and transfers it to the terminal. The terminal projects the new design proposal into the virtual space. This allows the user to experience the latest design proposal, which has been adjusted based on emotions, once again.

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0445] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0447] [Third Embodiment]

[0448] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0449] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0450] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0452] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0454] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0455] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0456] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0459] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0460] This invention provides a system that allows users to easily realize their ideal home design based on their lifestyle, family structure, hobbies, and future life plans, without requiring any specialized knowledge.

[0461] The server first receives attribute information provided by the user. This information includes detailed data that reflects the user's housing needs, such as lifestyle, preferences, and future plans. For example, suppose the server is given information that the user enjoys gardening and desires a large garden.

[0462] Subsequently, the server uses a generative model to analyze this attribute information. This analysis generates an initial house design proposal that matches the user's needs. The generated design proposal includes floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0463] The terminal receives the design proposal generated from the server and converts it into a three-dimensional model. Next, the terminal projects this three-dimensional model into a virtual space so that the user can experience a virtual tour. This allows the user to take a realistic tour as if they were actually walking through the house.

[0464] During the virtual tour, users can provide feedback on areas where the design needs adjustment. For example, they might want a larger kitchen or change the location of a window. User feedback is sent to the server in real time, and the server takes this feedback into consideration when revising the design.

[0465] The server uses AI agents to make adjustments to the design, taking into account market trends, regional characteristics, regulations, and environmental conditions. This process is repeated until the house design is completed in a way that satisfies the user. This system allows users to easily materialize their ideal living space and create a satisfying design.

[0466] The following describes the processing flow.

[0467] Step 1:

[0468] Users enter detailed information about their lifestyle, family structure, hobbies, and future life plans using a web form or a dedicated application. This information is stored on the device as attribute information necessary for housing design.

[0469] Step 2:

[0470] The terminal sends the collected attribute information to the server. This data is used as the basis for generating the design plan.

[0471] Step 3:

[0472] The server analyzes the received attribute information. Using a generative model, it automatically generates an initial house design proposal that best suits the user's needs. This design proposal includes floor plans and spatial arrangements that meet the user's specified requirements.

[0473] Step 4:

[0474] The server creates a three-dimensional model of the generated design data. This model is a detailed digital representation used later for virtual tours.

[0475] Step 5:

[0476] The device sets up the VR environment necessary for the user to conduct a virtual tour and projects a three-dimensional model into the virtual space, allowing the user to experience the interior of the house.

[0477] Step 6:

[0478] Users take a virtual tour of a house using a VR headset or compatible device, identifying aspects of the design and structure they dislike or would like to modify. They input this feedback into their device via voice or an interface during the virtual tour.

[0479] Step 7:

[0480] The device sends user feedback to the server. The server analyzes the received feedback and instructs the AI ​​agent to revise the design proposal.

[0481] Step 8:

[0482] The server remodels the design proposal, incorporating the feedback, into a 3D model and sends the updated model to the terminal. This process is repeated until the user is satisfied.

[0483] The above is the basic processing flow of the system, and through this series of steps, users can design a house that is as close to their ideal as possible.

[0484] (Example 1)

[0485] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0486] Traditionally, the residential design process has often required users to possess specialized knowledge, resulting in significant time and expense in realizing their ideal home. Furthermore, expert advice was essential for designing homes that take market trends and regional characteristics into account, which also represented a high hurdle for users. This invention solves these problems and provides a system that allows users to realize their ideal residential design without requiring specialized knowledge.

[0487] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0488] In this invention, the server includes means for collecting user attribute information through an input device and transmitting it as digital data, means for generating an initial design proposal using a generative AI model to analyze the user attribute information, and means for having a display device for converting the generated design proposal into a three-dimensional model and projecting it into a virtual space. This makes it possible for users to materialize a housing design that meets their needs while making real-time adjustments, even without specialized knowledge.

[0489] A "user" refers to an individual or organization that uses this system to provide their attribute information and engage in residential design.

[0490] "Attribute information" refers to information necessary for housing design, such as the user's lifestyle, family structure, hobbies, and future life plans.

[0491] A "generative AI model" is an artificial intelligence technology used to analyze user attribute information and generate optimal housing design proposals.

[0492] "Initial design proposal" refers to the prototype of a house design that is first created by a generative AI model based on the user's attribute information.

[0493] A "three-dimensional model" refers to a model created on a computer in a virtual three-dimensional space, allowing users to visually confirm design proposals.

[0494] A "virtual space" refers to a virtual environment created through computer simulation that is different from the real world.

[0495] "Feedback" refers to input information such as opinions, requests for adjustments, and suggestions for improvement that users provide regarding design proposals.

[0496] "Information processing means" refers to a combination of software and hardware used to analyze received digital data and make revisions to design proposals or new suggestions.

[0497] "Market trend information" refers to materials and data that show recent trends and user demand necessary for residential design.

[0498] "Regional characteristics information" refers to information that indicates the unique climate, culture, and building standards of a particular geographical area.

[0499] This system operates with a three-party relationship: user, server, and terminal. First, the user inputs attribute information such as their lifestyle, family structure, hobbies, and future life plans through the terminal's input device. This information is formatted as digital data and sent to the server. A concrete example of input would be something like, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0500] The server analyzes data using a generative AI model based on attribute information received from the user. This AI model plays a crucial role in creating initial housing design proposals that match the user's needs. The generated design proposals include floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0501] Subsequently, the terminal receives the design proposal sent from the server and uses existing 3D engines (e.g., Unity or Unreal Engine) as software to convert it into a three-dimensional model. This allows the terminal to project the design proposal into a virtual space, providing the user with a realistic virtual tour experience. Through this virtual tour, the user can experience what it's like to actually walk through the house. A concrete example of a prompt might be, "An eco-friendly house with a living room that comfortably accommodates a family of four."

[0502] During the virtual tour, users can provide feedback on the design. This feedback is sent to the server in real time via the terminal. For example, a user might input requests such as "I want a larger kitchen" or "I want more light in the living room." The server receives this feedback and uses the generated AI model and information processing device to revise the design proposal. This process, which considers market trends, regional characteristics, laws, and environmental conditions, is repeated to produce the optimal proposal, resulting in a house design that matches the user's ideal.

[0503] This system makes it easy for users to design a home that meets their needs, even without specialized knowledge.

[0504] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0505] Step 1:

[0506] Users use the input device on their terminal to enter attribute information such as their lifestyle, family structure, hobbies, and future life plans. The entered data is formatted as digital data and sent to the server. At this stage, users provide specific information such as, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0507] Step 2:

[0508] The server analyzes the digital data received from the user. A generative AI model is used for this analysis, and an initial design proposal is generated based on the user's attribute information. Based on the input attribute information, the generative AI model proposes floor plans and spatial arrangements, taking into account relevant design patterns and trends. As a result, a "3-bedroom design with a large garden and gardening space" is generated as the initial design proposal.

[0509] Step 3:

[0510] The server sends the generated initial design proposal to the terminal. The terminal then generates a 3D model based on this design proposal. Using a 3D engine (e.g., Unity or Unreal Engine), the design proposal is converted into a visual representation in a virtual space. The output is a 3D model that the user can visually recognize, and the virtual tour is ready.

[0511] Step 4:

[0512] The terminal uses the generated 3D model to provide users with a virtual tour. Users use display devices such as VR headsets to realistically experience the digitized interior of the house. Users can actually walk through and examine a "living room where a family of four can comfortably spend time," and get a feel for the design.

[0513] Step 5:

[0514] Users provide specific feedback during the virtual tour. For example, they can input requests such as "make the kitchen larger" via voice or text into their device. User requests are collected and sent to the server as feedback.

[0515] Step 6:

[0516] The server analyzes feedback received from the terminal and modifies the design proposal. It utilizes generative AI models and information processing tools to optimize the design proposal, taking into account the needs of new users. It updates the design by referring to databases of market trends, regional characteristics, laws, and environmental conditions. This process is repeated until the user is satisfied.

[0517] (Application Example 1)

[0518] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0519] It is difficult for users without specialized knowledge to realize their ideal home design that matches their lifestyle, family structure, hobbies, and future plans. Furthermore, there is a lack of timely means to reflect specific user feedback on design proposals and for users to easily and intuitively check the quality of the design using virtual tours. The challenge lies in quickly and flexibly revising design proposals to meet user needs.

[0520] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0521] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space; means for modifying the design proposals based on user feedback; means for the user to move around within the three-dimensional model during a virtual tour and grasp the details of specific parts; and means for collecting real-time feedback during movement and immediately modifying the design proposals. This enables users to realize an ideal house design that immediately reflects their needs and to provide feedback while experiencing the design process with intuitive operation.

[0522] "Attribute information" refers to data that indicates an individual's preferences and current situation, such as a user's lifestyle, family structure, hobbies, and future plans.

[0523] A "generative model" is an algorithm or program that uses a computer to generate housing design proposals based on a user's attribute information.

[0524] A "design proposal" refers to the initial design plan or floor plan of a house generated based on the user's needs.

[0525] A "three-dimensional model" is a digital representation of a virtual space expressed within a computer-generated three-dimensional space.

[0526] "Virtual tour" is a process in which users explore a three-dimensional model in a virtual space through a digital device and visually confirm the design.

[0527] "Feedback" refers to the input of opinions and requests for revisions that users provide regarding the design proposal, and this information is reflected in the design by the system.

[0528] "Real-time" refers to a time concept where responses and processing occur immediately in response to user input or actions.

[0529] A description of the embodiment for carrying out the invention will be provided.

[0530] In this system, the user, the server, and the terminal play three main roles.

[0531] Users enter their personal attribute information into the application. This includes information about their lifestyle, family structure, hobbies, and future plans, and this information forms the basis of the system.

[0532] The server uses the received attribute information to generate an initial house design proposal tailored to the user, utilizing a generative AI model. This generative AI model can employ advanced algorithms such as GPT-3. The generated design proposal includes floor plans and digital designs based on the user's needs.

[0533] Subsequently, the terminal receives the design proposal from the server and projects it into the virtual space as a three-dimensional model. Suitable platforms for this are three-dimensional modeling engines such as Unity and Unreal Engine. The terminal utilizes devices such as smartphones or head-mounted displays (e.g., Oculus Quest) to provide the user with a virtual tour experience.

[0534] During the virtual tour, users can freely move around the virtual space, examine details, and provide feedback. For example, they can send feedback such as "I want a bigger kitchen" via voice input or touch controls, and this information is immediately sent to the server and incorporated into the design.

[0535] The server processes feedback in real time, modifies the design proposal, and then sends the updated 3D model back to the terminal. This process allows users to intuitively and efficiently realize their ideal home design.

[0536] For example, if a user enters a prompt message such as "I want to make the living room window a little bigger," the server will immediately modify the design proposal based on that request and provide the user with an updated 3D model.

[0537] Examples of prompt statements are as follows:

[0538] User: I'd like to make the living room window a little bigger.

[0539] System: We have received user feedback. We are adjusting the window design.

[0540] User: I want to expand my kitchen space.

[0541] System: A new design proposal has been displayed. Please see this kitchen space.

[0542] This allows users to quickly and flexibly experience home designs that reflect their needs and intuitively make the necessary changes.

[0543] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0544] Step 1:

[0545] Users input attribute information such as lifestyle, family structure, hobbies, and future plans into a terminal. This input information is sent from the terminal to the server. This input data forms the basis for subsequent analysis by a generative model.

[0546] Step 2:

[0547] The server performs analysis using a generated AI model (e.g., GPT-3) based on the received user attribute information. This analysis extracts design requirements that meet the user's needs and generates an initial design proposal for a house. The output design proposal includes digital data such as floor plans and spatial designs.

[0548] Step 3:

[0549] The server sends the generated design proposal to the terminal. The terminal converts the received design proposal into a three-dimensional model. This conversion uses a three-dimensional modeling engine such as Unity or Unreal Engine. The output is a three-dimensional model in a virtual space that the user can visually confirm.

[0550] Step 4:

[0551] Users conduct virtual tours using their devices. They can freely move around within the 3D model and provide feedback on specific designs through the interface. For example, users can input prompts such as "I want to make the kitchen bigger" using voice input or touch controls.

[0552] Step 5:

[0553] The terminal collects user feedback and sends it to the server. The server processes this feedback in real time and performs data processing to incorporate it into the design proposal. Specifically, it analyzes the feedback content and makes design changes to the areas that were pointed out.

[0554] Step 6:

[0555] The server sends the revised design proposal back to the terminal. This updated design proposal is converted into a new 3D model and displayed to the user. The user takes another virtual tour and provides further feedback if necessary.

[0556] This series of processes allows users to efficiently realize their ideal living space.

[0557] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0558] This invention incorporates an emotion engine into a system that automates home design based on a user's lifestyle, family structure, hobbies, and future plans. The emotion engine recognizes the user's emotions and optimizes the design accordingly.

[0559] The server first receives attribute information from the user. This attribute information includes the user's requests and expectations regarding the house design. The server uses a generative model to analyze the user's data and generate initial house design proposals.

[0560] Next, the server creates a 3D model of the generated design proposal and projects it into a virtual space via the terminal. The terminal then uses a VR device to provide this virtual space to the user, allowing the user to actually view the design proposal.

[0561] In addition, this system is equipped with an emotion engine. The terminal analyzes the user's facial expressions and voice to recognize their current emotional state. For example, if the emotion engine detects facial expressions indicating dissatisfaction or surprise while the user is taking a virtual tour, it sends this information to the server. Based on this emotional data, the server adjusts the design to correspond to the user's emotional state. For example, it might suggest design changes that emphasize areas where the user is feeling happy.

[0562] User feedback based on emotions is processed on the server in real time. The AI ​​agent then modifies the design proposal based on market trends, regional characteristics, laws, and environmental conditions, and presents the 3D model to the user again.

[0563] A key feature of this system is that it provides an intuitive design process that takes user emotions into consideration, enabling efficient guidance to arrive at the optimal design for the user. Through emotion-based feedback, users can achieve a more satisfying home design.

[0564] The following describes the processing flow.

[0565] Step 1:

[0566] Users input information about their lifestyle, family structure, hobbies, and future plans through their devices. This allows for the collection of user attribute information.

[0567] Step 2:

[0568] The terminal sends the collected attribute information to the server. The server then prepares to analyze this information.

[0569] Step 3:

[0570] The server uses a generative model to analyze the user's attribute information and generate an initial house design proposal. This design proposal is customized based on the user's requests.

[0571] Step 4:

[0572] The server converts the generated design proposal into a three-dimensional model. This model is later used for projection in the virtual space.

[0573] Step 5:

[0574] The terminal projects the 3D model received from the server into a virtual space. The user can use a VR device to freely move around in this space and virtually tour the design proposal.

[0575] Step 6:

[0576] During a virtual tour, the system analyzes the user's facial expressions and voice through the device's camera and microphone using an emotion engine. This allows for real-time recognition of the user's emotional state.

[0577] Step 7:

[0578] The emotion engine recognizes the user's emotions, which are then sent from the device to the server. The server uses this emotion data to revise the design proposal.

[0579] Step 8:

[0580] The server uses an AI agent to adjust design proposals based on the user's emotions. For example, if a user expresses pleasure with a particular design, it will generate a revised proposal that emphasizes that element.

[0581] Step 9:

[0582] The server generates the adjusted design proposal again as a 3D model and sends it to the terminal.

[0583] Step 10:

[0584] The device projects the updated 3D model into a virtual space, presenting the user with new design proposals. This process is repeated until the user is satisfied with the design.

[0585] In this way, design proposals that reflect the user's feelings are made and revised, enabling the user to realize the optimal housing design.

[0586] (Example 2)

[0587] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] Conventional housing design systems struggle to optimize designs by considering user attribute information and emotions, lacking a design process that enhances user emotional satisfaction. Furthermore, they struggle to provide designs that effectively reflect market trends, regional characteristics, laws, and environmental conditions. This creates a challenge in quickly and efficiently proposing homes that truly meet user needs.

[0589] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0590] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model, means for projecting the generated design proposals as three-dimensional models into a virtual space, and means for analyzing the user's facial expressions and voice to recognize their emotional state. This makes it possible to optimize the design according to the user's emotions and provide design proposals that take into account market trends, regional characteristics, laws and regulations, and environmental conditions.

[0591] "User attribute information" refers to information that includes individual needs and expectations, such as the user's lifestyle, family structure, hobbies, and future plans.

[0592] "Methods for generating design proposals based on generative models" refer to methods that utilize collected attribute information and automatically generate residential design proposals using artificial intelligence and machine learning technologies.

[0593] "A means of projecting a three-dimensional model into a virtual space" refers to a technology that visualizes the generated design proposal three-dimensionally as digital data and projects it into a virtual environment that users can visually confirm.

[0594] "Means for analyzing a user's facial expressions and voice to recognize their emotional state" refers to a method of analyzing a user's facial expressions and voice tone using sensors and analysis software to identify the user's emotional response.

[0595] "Optimization" refers to the process of improving design proposals based on user attribute information and emotional states, and adjusting the design to enhance user satisfaction.

[0596] "Market trends, regional characteristics, laws, and environmental conditions" refers to current market needs, unique regional characteristics, relevant laws and regulations, and environmental constraints and considerations.

[0597] This invention provides a system that automates house design based on user attribute information, and optimizes the design by taking into account the user's emotional state during the process. The system mainly consists of a server, terminals, and user interaction.

[0598] First, the server receives attribute information from the user regarding their lifestyle, family structure, hobbies, future plans, etc. This information reflects the user's specific requests and expectations for their home design. Based on the collected attribute information, the server uses a generative AI model to generate initial home design proposals. This generative AI model takes specific prompt statements as input and outputs design proposals. An example of a prompt statement would be, "Please recommend a home design for a single musician in their 30s, including an open music room that incorporates plenty of natural light."

[0599] Next, the server converts the generated design proposal into a three-dimensional model using 3D modeling software. This 3D model is represented in a virtual space designed to allow users to virtually tour the interior. The completed 3D model is sent to a terminal, and users can experience it using a VR device.

[0600] The device is equipped with a camera and microphone to analyze the user's facial expressions and voice during the virtual tour, and uses an emotion engine to recognize emotional states in real time. This process monitors the user's facial expressions and voice tone in real time and determines which emotion they correspond to, such as joy, surprise, or dissatisfaction.

[0601] The recognized emotion information is sent to the server, which optimizes the design proposal based on the user's emotional state. For example, areas where the user felt joy are highlighted or made more appealing in the design suggestions. Furthermore, an AI agent is responsible for revising the design proposal, taking into account market trends, regional characteristics, laws and regulations, and environmental conditions, and then re-transmitting the improved 3D model to the terminal.

[0602] This system allows users to provide emotional feedback, enabling more personalized designs and ultimately increasing satisfaction with the home design.

[0603] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0604] Step 1:

[0605] The server receives attribute information from the user as input. Specifically, the user enters information about their lifestyle, family structure, hobbies, and future plans. This information is stored in a database and digitized for use in subsequent processing.

[0606] Step 2:

[0607] The server inputs the received attribute information as a prompt to the generating AI model. An example of a prompt is, "Please propose a design for a 2LDK apartment including a gardening space." Based on this input, the generating AI model generates an initial residential design and outputs it as design data.

[0608] Step 3:

[0609] The server takes the generated design proposal as input and converts it into a three-dimensional model using 3D modeling software. Specifically, it renders each element of the design proposal into a three-dimensional shape and outputs it in a format suitable for the virtual space.

[0610] Step 4:

[0611] The terminal receives a 3D model sent from the server and projects it into a virtual space using a VR device. Users can directly view the design proposal and experience its interior within the VR space.

[0612] Step 5:

[0613] Users take a virtual tour through a VR device, and their facial expressions and voice are recorded in real time by the device's camera and microphone. This data is treated as input data indicating the user's emotions.

[0614] Step 6:

[0615] The device uses an emotion engine to analyze input facial and voice data and output the user's emotional state. Specifically, it uses facial recognition algorithms and voice analysis to identify emotions such as joy, surprise, and dissatisfaction.

[0616] Step 7:

[0617] The server takes the analyzed sentiment data as input and optimizes the design proposal. Based on the sentiment data, it suggests design changes, such as highlighting aspects that users found favorable, and generates the revised design data.

[0618] Step 8:

[0619] The server uses an AI agent to further refine the optimized design proposal, taking into account market trends, regional characteristics, laws, and environmental conditions. This results in the output of a final design proposal that conforms to laws and local regulations.

[0620] Step 9:

[0621] The server then sends the 3D model of the final design proposal back to the terminal, presenting the revised proposal to the user. This allows the user to review the updated design proposal and evaluate the differences.

[0622] (Application Example 2)

[0623] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0624] In residential design, providing an individually optimized living environment that considers not only residents' attribute information but also their emotional state has been difficult with conventional technologies. There is a need for methods that can significantly improve resident satisfaction by analyzing users' emotions in real time and providing experiences that align with those emotions.

[0625] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0626] In this invention, the server includes means for collecting user attribute information and emotional state and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space and modifying the design proposals based on emotional feedback; and means for analyzing the user's emotions in the virtual space in real time using emotion recognition technology and reflecting these in the design proposals. This makes it possible to provide users with design proposals optimized for their needs and to realize an experience that aligns with their emotions.

[0627] A "user" is an individual who uses this system to request a house design.

[0628] "Attribute information" refers to data such as the user's lifestyle, family structure, hobbies, and future plans.

[0629] "Emotional state" refers to information about the user's psychological response, extracted from their facial expressions, voice, and other similar data.

[0630] A "generative model" is an algorithm that generates design proposals based on user attribute information and emotional state.

[0631] A "design proposal" is an initial plan for a house that takes into account the user's needs and feelings.

[0632] A "three-dimensional model" is a three-dimensional data representation that visually shows a design proposal in a virtual space.

[0633] A "virtual space" is a digital environment created by a computer system that is different from reality.

[0634] "Emotional feedback" refers to response information based on the user's emotional state.

[0635] "Emotion recognition technology" refers to technical methods for analyzing a user's emotional state.

[0636] "Real-time" refers to a time concept that describes the process by which a user's emotional state is analyzed and reflected immediately.

[0637] The system implementing this invention generates and presents individually optimized housing designs in a virtual space based on attribute information such as the user's lifestyle, family structure, hobbies, and future plans, as well as emotional states obtained from facial expressions and voice using emotion recognition technology. The server collects the user's attribute information and generates design proposals using a generation AI model. The generated design proposals are projected as three-dimensional models into a virtual space via a VR device on the terminal. The user experiences the design proposals in this virtual space, and their emotional state is analyzed in real time using emotion recognition technology. Based on the emotional feedback, the server modifies the design proposals and provides the user with optimized designs. This makes it possible to realize an experience that aligns with emotions and achieve highly satisfying housing designs.

[0638] Specifically, the device uses an emotion recognition API to analyze the user's emotional state in real time through its camera and microphone. The server uses a generative AI model to generate new design proposals based on the collected attribute information and emotional state. The regenerated design proposals are then converted into 3D models and presented to the user in a virtual space on the device. The software used includes, for example, 3D engines such as Unity and Unreal Engine, and cloud services such as Amazon Web Services and Google Cloud Platform.

[0639] For example, if a user expresses emotions such as "surprise" or "joy" while viewing the living room design during a virtual tour, the system will use this reaction to suggest ways to further enhance the living room design. An example of a prompt might be, "If the user is enjoying the living room, suggest ways to further enhance its design."

[0640] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0641] Step 1:

[0642] The server receives attribute information from the user. As input, it collects data such as lifestyle, family structure, hobbies, and future plans. This provides the basic information needed to generate design proposals optimized for the user.

[0643] Step 2:

[0644] The server inputs the collected attribute information into a generating AI model to generate design proposals. The generating AI model extracts the necessary conditions for the design from the attribute information and outputs a design proposal based on those conditions. This creates a personalized house design proposal for the user.

[0645] Step 3:

[0646] The terminal projects the generated design proposal as a three-dimensional model into the virtual space. Using the design proposal received from the server as input, it creates a visual model that the user can actually experience using a 3D engine. This allows the user to virtually tour the design proposal.

[0647] Step 4:

[0648] Users explore a virtual space through a VR device. This allows users to review design proposals and intuitively grasp the layout and arrangement of each room. As a result, users can develop genuine opinions and feelings about the design.

[0649] Step 5:

[0650] The device uses emotion recognition technology to analyze the user's facial expressions and voice, detecting their emotional state in real time. It acquires the user's camera video and audio data as input and processes it using an emotion recognition API. This allows it to acquire and analyze current emotional data based on the user's reactions.

[0651] Step 6:

[0652] The server modifies the design proposal based on the detected emotional state. The AI ​​model receives emotional data as input and recalculates the design proposal, outputting it as an output. This ensures that the design elements that should be emphasized are reflected according to the user's emotions.

[0653] Step 7:

[0654] The server then remodels the revised design proposal into a 3D model and transfers it to the terminal. The terminal projects the new design proposal into the virtual space. This allows the user to experience the latest design proposal, which has been adjusted based on emotions, once again.

[0655] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0656] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0657] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0658] [Fourth Embodiment]

[0659] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0660] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0661] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0662] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0663] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0664] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0665] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0666] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0667] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0668] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0669] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0670] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0671] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0672] This invention provides a system that allows users to easily realize their ideal home design based on their lifestyle, family structure, hobbies, and future life plans, without requiring any specialized knowledge.

[0673] The server first receives attribute information provided by the user. This information includes detailed data that reflects the user's housing needs, such as lifestyle, preferences, and future plans. For example, suppose the server is given information that the user enjoys gardening and desires a large garden.

[0674] Subsequently, the server uses a generative model to analyze this attribute information. This analysis generates an initial house design proposal that matches the user's needs. The generated design proposal includes floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0675] The terminal receives the design proposal generated from the server and converts it into a three-dimensional model. Next, the terminal projects this three-dimensional model into a virtual space so that the user can experience a virtual tour. This allows the user to take a realistic tour as if they were actually walking through the house.

[0676] During the virtual tour, users can provide feedback on areas where the design needs adjustment. For example, they might want a larger kitchen or change the location of a window. User feedback is sent to the server in real time, and the server takes this feedback into consideration when revising the design.

[0677] The server uses AI agents to make adjustments to the design, taking into account market trends, regional characteristics, regulations, and environmental conditions. This process is repeated until the house design is completed in a way that satisfies the user. This system allows users to easily materialize their ideal living space and create a satisfying design.

[0678] The following describes the processing flow.

[0679] Step 1:

[0680] Users enter detailed information about their lifestyle, family structure, hobbies, and future life plans using a web form or a dedicated application. This information is stored on the device as attribute information necessary for housing design.

[0681] Step 2:

[0682] The terminal sends the collected attribute information to the server. This data is used as the basis for generating the design plan.

[0683] Step 3:

[0684] The server analyzes the received attribute information. Using a generative model, it automatically generates an initial house design proposal that best suits the user's needs. This design proposal includes floor plans and spatial arrangements that meet the user's specified requirements.

[0685] Step 4:

[0686] The server creates a three-dimensional model of the generated design data. This model is a detailed digital representation used later for virtual tours.

[0687] Step 5:

[0688] The device sets up the VR environment necessary for the user to conduct a virtual tour and projects a three-dimensional model into the virtual space, allowing the user to experience the interior of the house.

[0689] Step 6:

[0690] Users take a virtual tour of a house using a VR headset or compatible device, identifying aspects of the design and structure they dislike or would like to modify. They input this feedback into their device via voice or an interface during the virtual tour.

[0691] Step 7:

[0692] The device sends user feedback to the server. The server analyzes the received feedback and instructs the AI ​​agent to revise the design proposal.

[0693] Step 8:

[0694] The server remodels the design proposal, incorporating the feedback, into a 3D model and sends the updated model to the terminal. This process is repeated until the user is satisfied.

[0695] The above is the basic processing flow of the system, and through this series of steps, users can design a house that is as close to their ideal as possible.

[0696] (Example 1)

[0697] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0698] Traditionally, the residential design process has often required users to possess specialized knowledge, resulting in significant time and expense in realizing their ideal home. Furthermore, expert advice was essential for designing homes that take market trends and regional characteristics into account, which also represented a high hurdle for users. This invention solves these problems and provides a system that allows users to realize their ideal residential design without requiring specialized knowledge.

[0699] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0700] In this invention, the server includes means for collecting user attribute information through an input device and transmitting it as digital data, means for generating an initial design proposal using a generative AI model to analyze the user attribute information, and means for having a display device for converting the generated design proposal into a three-dimensional model and projecting it into a virtual space. This makes it possible for users to materialize a housing design that meets their needs while making real-time adjustments, even without specialized knowledge.

[0701] A "user" refers to an individual or organization that uses this system to provide their attribute information and engage in residential design.

[0702] "Attribute information" refers to information necessary for housing design, such as the user's lifestyle, family structure, hobbies, and future life plans.

[0703] A "generative AI model" is an artificial intelligence technology used to analyze user attribute information and generate optimal housing design proposals.

[0704] "Initial design proposal" refers to the prototype of a house design that is first created by a generative AI model based on the user's attribute information.

[0705] A "three-dimensional model" refers to a model created on a computer in a virtual three-dimensional space, allowing users to visually confirm design proposals.

[0706] A "virtual space" refers to a virtual environment created through computer simulation that is different from the real world.

[0707] "Feedback" refers to input information such as opinions, requests for adjustments, and suggestions for improvement that users provide regarding design proposals.

[0708] "Information processing means" refers to a combination of software and hardware used to analyze received digital data and make revisions to design proposals or new suggestions.

[0709] "Market trend information" refers to materials and data that show recent trends and user demand necessary for residential design.

[0710] "Regional characteristics information" refers to information that indicates the unique climate, culture, and building standards of a particular geographical area.

[0711] This system operates with a three-party relationship: user, server, and terminal. First, the user inputs attribute information such as their lifestyle, family structure, hobbies, and future life plans through the terminal's input device. This information is formatted as digital data and sent to the server. A concrete example of input would be something like, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0712] The server analyzes data using a generative AI model based on attribute information received from the user. This AI model plays a crucial role in creating initial housing design proposals that match the user's needs. The generated design proposals include floor plans, spatial arrangements, and design elements that align with the user's preferences.

[0713] Subsequently, the terminal receives the design proposal sent from the server and uses existing 3D engines (e.g., Unity or Unreal Engine) as software to convert it into a three-dimensional model. This allows the terminal to project the design proposal into a virtual space, providing the user with a realistic virtual tour experience. Through this virtual tour, the user can experience what it's like to actually walk through the house. A concrete example of a prompt might be, "An eco-friendly house with a living room that comfortably accommodates a family of four."

[0714] During the virtual tour, users can provide feedback on the design. This feedback is sent to the server in real time via the terminal. For example, a user might input requests such as "I want a larger kitchen" or "I want more light in the living room." The server receives this feedback and uses the generated AI model and information processing device to revise the design proposal. This process, which considers market trends, regional characteristics, laws, and environmental conditions, is repeated to produce the optimal proposal, resulting in a house design that matches the user's ideal.

[0715] This system makes it easy for users to design a home that meets their needs, even without specialized knowledge.

[0716] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0717] Step 1:

[0718] Users use the input device on their terminal to enter attribute information such as their lifestyle, family structure, hobbies, and future life plans. The entered data is formatted as digital data and sent to the server. At this stage, users provide specific information such as, "I enjoy gardening, I plan to have a family of three in the future, and I want a large garden."

[0719] Step 2:

[0720] The server analyzes the digital data received from the user. A generative AI model is used for this analysis, and an initial design proposal is generated based on the user's attribute information. Based on the input attribute information, the generative AI model proposes floor plans and spatial arrangements, taking into account relevant design patterns and trends. As a result, a "3-bedroom design with a large garden and gardening space" is generated as the initial design proposal.

[0721] Step 3:

[0722] The server sends the generated initial design proposal to the terminal. The terminal then generates a 3D model based on this design proposal. Using a 3D engine (e.g., Unity or Unreal Engine), the design proposal is converted into a visual representation in a virtual space. The output is a 3D model that the user can visually recognize, and the virtual tour is ready.

[0723] Step 4:

[0724] The terminal uses the generated 3D model to provide users with a virtual tour. Users use display devices such as VR headsets to realistically experience the digitized interior of the house. Users can actually walk through and examine a "living room where a family of four can comfortably spend time," and get a feel for the design.

[0725] Step 5:

[0726] Users provide specific feedback during the virtual tour. For example, they can input requests such as "make the kitchen larger" via voice or text into their device. User requests are collected and sent to the server as feedback.

[0727] Step 6:

[0728] The server analyzes feedback received from the terminal and modifies the design proposal. It utilizes generative AI models and information processing tools to optimize the design proposal, taking into account the needs of new users. It updates the design by referring to databases of market trends, regional characteristics, laws, and environmental conditions. This process is repeated until the user is satisfied.

[0729] (Application Example 1)

[0730] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0731] It is difficult for users without specialized knowledge to realize their ideal home design that matches their lifestyle, family structure, hobbies, and future plans. Furthermore, there is a lack of timely means to reflect specific user feedback on design proposals and for users to easily and intuitively check the quality of the design using virtual tours. The challenge lies in quickly and flexibly revising design proposals to meet user needs.

[0732] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0733] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space; means for modifying the design proposals based on user feedback; means for the user to move around within the three-dimensional model during a virtual tour and grasp the details of specific parts; and means for collecting real-time feedback during movement and immediately modifying the design proposals. This enables users to realize an ideal house design that immediately reflects their needs and to provide feedback while experiencing the design process with intuitive operation.

[0734] "Attribute information" refers to data that indicates an individual's preferences and current situation, such as a user's lifestyle, family structure, hobbies, and future plans.

[0735] A "generative model" is an algorithm or program that uses a computer to generate housing design proposals based on a user's attribute information.

[0736] A "design proposal" refers to the initial design plan or floor plan of a house generated based on the user's needs.

[0737] A "three-dimensional model" is a digital representation of a virtual space expressed within a computer-generated three-dimensional space.

[0738] "Virtual tour" is a process in which users explore a three-dimensional model in a virtual space through a digital device and visually confirm the design.

[0739] "Feedback" refers to the input of opinions and requests for revisions that users provide regarding the design proposal, and this information is reflected in the design by the system.

[0740] "Real-time" refers to a time concept where responses and processing occur immediately in response to user input or actions.

[0741] A description of the embodiment for carrying out the invention will be provided.

[0742] In this system, the user, the server, and the terminal play three main roles.

[0743] Users enter their personal attribute information into the application. This includes information about their lifestyle, family structure, hobbies, and future plans, and this information forms the basis of the system.

[0744] The server uses the received attribute information to generate an initial house design proposal tailored to the user, utilizing a generative AI model. This generative AI model can employ advanced algorithms such as GPT-3. The generated design proposal includes floor plans and digital designs based on the user's needs.

[0745] Subsequently, the terminal receives the design proposal from the server and projects it into the virtual space as a three-dimensional model. Suitable platforms for this are three-dimensional modeling engines such as Unity and Unreal Engine. The terminal utilizes devices such as smartphones or head-mounted displays (e.g., Oculus Quest) to provide the user with a virtual tour experience.

[0746] During the virtual tour, users can freely move around the virtual space, examine details, and provide feedback. For example, they can send feedback such as "I want a bigger kitchen" via voice input or touch controls, and this information is immediately sent to the server and incorporated into the design.

[0747] The server processes feedback in real time, modifies the design proposal, and then sends the updated 3D model back to the terminal. This process allows users to intuitively and efficiently realize their ideal home design.

[0748] For example, if a user enters a prompt message such as "I want to make the living room window a little bigger," the server will immediately modify the design proposal based on that request and provide the user with an updated 3D model.

[0749] Examples of prompt statements are as follows:

[0750] User: I'd like to make the living room window a little bigger.

[0751] System: We have received user feedback. We are adjusting the window design.

[0752] User: I want to expand my kitchen space.

[0753] System: A new design proposal has been displayed. Please see this kitchen space.

[0754] This allows users to quickly and flexibly experience home designs that reflect their needs and intuitively make the necessary changes.

[0755] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0756] Step 1:

[0757] Users input attribute information such as lifestyle, family structure, hobbies, and future plans into a terminal. This input information is sent from the terminal to the server. This input data forms the basis for subsequent analysis by a generative model.

[0758] Step 2:

[0759] The server performs analysis using a generated AI model (e.g., GPT-3) based on the received user attribute information. This analysis extracts design requirements that meet the user's needs and generates an initial design proposal for a house. The output design proposal includes digital data such as floor plans and spatial designs.

[0760] Step 3:

[0761] The server sends the generated design proposal to the terminal. The terminal converts the received design proposal into a three-dimensional model. This conversion uses a three-dimensional modeling engine such as Unity or Unreal Engine. The output is a three-dimensional model in a virtual space that the user can visually confirm.

[0762] Step 4:

[0763] Users conduct virtual tours using their devices. They can freely move around within the 3D model and provide feedback on specific designs through the interface. For example, users can input prompts such as "I want to make the kitchen bigger" using voice input or touch controls.

[0764] Step 5:

[0765] The terminal collects user feedback and sends it to the server. The server processes this feedback in real time and performs data processing to incorporate it into the design proposal. Specifically, it analyzes the feedback content and makes design changes to the areas that were pointed out.

[0766] Step 6:

[0767] The server sends the revised design proposal back to the terminal. This updated design proposal is converted into a new 3D model and displayed to the user. The user takes another virtual tour and provides further feedback if necessary.

[0768] This series of processes allows users to efficiently realize their ideal living space.

[0769] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0770] This invention incorporates an emotion engine into a system that automates home design based on a user's lifestyle, family structure, hobbies, and future plans. The emotion engine recognizes the user's emotions and optimizes the design accordingly.

[0771] The server first receives attribute information from the user. This attribute information includes the user's requests and expectations regarding the house design. The server uses a generative model to analyze the user's data and generate initial house design proposals.

[0772] Next, the server creates a 3D model of the generated design proposal and projects it into a virtual space via the terminal. The terminal then uses a VR device to provide this virtual space to the user, allowing the user to actually view the design proposal.

[0773] In addition, this system is equipped with an emotion engine. The terminal analyzes the user's facial expressions and voice to recognize their current emotional state. For example, if the emotion engine detects facial expressions indicating dissatisfaction or surprise while the user is taking a virtual tour, it sends this information to the server. Based on this emotional data, the server adjusts the design to correspond to the user's emotional state. For example, it might suggest design changes that emphasize areas where the user is feeling happy.

[0774] User feedback based on emotions is processed on the server in real time. The AI ​​agent then modifies the design proposal based on market trends, regional characteristics, laws, and environmental conditions, and presents the 3D model to the user again.

[0775] A key feature of this system is that it provides an intuitive design process that takes user emotions into consideration, enabling efficient guidance to arrive at the optimal design for the user. Through emotion-based feedback, users can achieve a more satisfying home design.

[0776] The following describes the processing flow.

[0777] Step 1:

[0778] Users input information about their lifestyle, family structure, hobbies, and future plans through their devices. This allows for the collection of user attribute information.

[0779] Step 2:

[0780] The terminal sends the collected attribute information to the server. The server then prepares to analyze this information.

[0781] Step 3:

[0782] The server uses a generative model to analyze the user's attribute information and generate an initial house design proposal. This design proposal is customized based on the user's requests.

[0783] Step 4:

[0784] The server converts the generated design proposal into a three-dimensional model. This model is later used for projection in the virtual space.

[0785] Step 5:

[0786] The terminal projects the 3D model received from the server into a virtual space. The user can use a VR device to freely move around in this space and virtually tour the design proposal.

[0787] Step 6:

[0788] During a virtual tour, the system analyzes the user's facial expressions and voice through the device's camera and microphone using an emotion engine. This allows for real-time recognition of the user's emotional state.

[0789] Step 7:

[0790] The emotion engine recognizes the user's emotions, which are then sent from the device to the server. The server uses this emotion data to revise the design proposal.

[0791] Step 8:

[0792] The server uses an AI agent to adjust design proposals based on the user's emotions. For example, if a user expresses pleasure with a particular design, it will generate a revised proposal that emphasizes that element.

[0793] Step 9:

[0794] The server generates the adjusted design proposal again as a 3D model and sends it to the terminal.

[0795] Step 10:

[0796] The device projects the updated 3D model into a virtual space, presenting the user with new design proposals. This process is repeated until the user is satisfied with the design.

[0797] In this way, design proposals that reflect the user's feelings are made and revised, enabling the user to realize the optimal housing design.

[0798] (Example 2)

[0799] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0800] Conventional housing design systems struggle to optimize designs by considering user attribute information and emotions, lacking a design process that enhances user emotional satisfaction. Furthermore, they struggle to provide designs that effectively reflect market trends, regional characteristics, laws, and environmental conditions. This creates a challenge in quickly and efficiently proposing homes that truly meet user needs.

[0801] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0802] In this invention, the server includes means for collecting user attribute information and generating design proposals based on a generative model, means for projecting the generated design proposals as three-dimensional models into a virtual space, and means for analyzing the user's facial expressions and voice to recognize their emotional state. This makes it possible to optimize the design according to the user's emotions and provide design proposals that take into account market trends, regional characteristics, laws and regulations, and environmental conditions.

[0803] "User attribute information" refers to information that includes individual needs and expectations, such as the user's lifestyle, family structure, hobbies, and future plans.

[0804] "Methods for generating design proposals based on generative models" refer to methods that utilize collected attribute information and automatically generate residential design proposals using artificial intelligence and machine learning technologies.

[0805] "A means of projecting a three-dimensional model into a virtual space" refers to a technology that visualizes the generated design proposal three-dimensionally as digital data and projects it into a virtual environment that users can visually confirm.

[0806] "Means for analyzing a user's facial expressions and voice to recognize their emotional state" refers to a method of analyzing a user's facial expressions and voice tone using sensors and analysis software to identify the user's emotional response.

[0807] "Optimization" refers to the process of improving design proposals based on user attribute information and emotional states, and adjusting the design to enhance user satisfaction.

[0808] "Market trends, regional characteristics, laws, and environmental conditions" refers to current market needs, unique regional characteristics, relevant laws and regulations, and environmental constraints and considerations.

[0809] This invention provides a system that automates house design based on user attribute information, and optimizes the design by taking into account the user's emotional state during the process. The system mainly consists of a server, terminals, and user interaction.

[0810] First, the server receives attribute information from the user regarding their lifestyle, family structure, hobbies, future plans, etc. This information reflects the user's specific requests and expectations for their home design. Based on the collected attribute information, the server uses a generative AI model to generate initial home design proposals. This generative AI model takes specific prompt statements as input and outputs design proposals. An example of a prompt statement would be, "Please recommend a home design for a single musician in their 30s, including an open music room that incorporates plenty of natural light."

[0811] Next, the server converts the generated design proposal into a three-dimensional model using 3D modeling software. This 3D model is represented in a virtual space designed to allow users to virtually tour the interior. The completed 3D model is sent to a terminal, and users can experience it using a VR device.

[0812] The device is equipped with a camera and microphone to analyze the user's facial expressions and voice during the virtual tour, and uses an emotion engine to recognize emotional states in real time. This process monitors the user's facial expressions and voice tone in real time and determines which emotion they correspond to, such as joy, surprise, or dissatisfaction.

[0813] The recognized emotion information is sent to the server, which optimizes the design proposal based on the user's emotional state. For example, areas where the user felt joy are highlighted or made more appealing in the design suggestions. Furthermore, an AI agent is responsible for revising the design proposal, taking into account market trends, regional characteristics, laws and regulations, and environmental conditions, and then re-transmitting the improved 3D model to the terminal.

[0814] This system allows users to provide emotional feedback, enabling more personalized designs and ultimately increasing satisfaction with the home design.

[0815] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0816] Step 1:

[0817] The server receives attribute information from the user as input. Specifically, the user enters information about their lifestyle, family structure, hobbies, and future plans. This information is stored in a database and digitized for use in subsequent processing.

[0818] Step 2:

[0819] The server inputs the received attribute information as a prompt to the generating AI model. An example of a prompt is, "Please propose a design for a 2LDK apartment including a gardening space." Based on this input, the generating AI model generates an initial residential design and outputs it as design data.

[0820] Step 3:

[0821] The server takes the generated design proposal as input and converts it into a three-dimensional model using 3D modeling software. Specifically, it renders each element of the design proposal into a three-dimensional shape and outputs it in a format suitable for the virtual space.

[0822] Step 4:

[0823] The terminal receives a 3D model sent from the server and projects it into a virtual space using a VR device. Users can directly view the design proposal and experience its interior within the VR space.

[0824] Step 5:

[0825] Users take a virtual tour through a VR device, and their facial expressions and voice are recorded in real time by the device's camera and microphone. This data is treated as input data indicating the user's emotions.

[0826] Step 6:

[0827] The device uses an emotion engine to analyze input facial and voice data and output the user's emotional state. Specifically, it uses facial recognition algorithms and voice analysis to identify emotions such as joy, surprise, and dissatisfaction.

[0828] Step 7:

[0829] The server takes the analyzed sentiment data as input and optimizes the design proposal. Based on the sentiment data, it suggests design changes, such as highlighting aspects that users found favorable, and generates the revised design data.

[0830] Step 8:

[0831] The server uses an AI agent to further refine the optimized design proposal, taking into account market trends, regional characteristics, laws, and environmental conditions. This results in the output of a final design proposal that conforms to laws and local regulations.

[0832] Step 9:

[0833] The server then sends the 3D model of the final design proposal back to the terminal, presenting the revised proposal to the user. This allows the user to review the updated design proposal and evaluate the differences.

[0834] (Application Example 2)

[0835] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0836] In residential design, providing an individually optimized living environment that considers not only residents' attribute information but also their emotional state has been difficult with conventional technologies. There is a need for methods that can significantly improve resident satisfaction by analyzing users' emotions in real time and providing experiences that align with those emotions.

[0837] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0838] In this invention, the server includes means for collecting user attribute information and emotional state and generating design proposals based on a generative model; means for projecting the generated design proposals as three-dimensional models into a virtual space and modifying the design proposals based on emotional feedback; and means for analyzing the user's emotions in the virtual space in real time using emotion recognition technology and reflecting these in the design proposals. This makes it possible to provide users with design proposals optimized for their needs and to realize an experience that aligns with their emotions.

[0839] A "user" is an individual who uses this system to request a house design.

[0840] "Attribute information" refers to data such as the user's lifestyle, family structure, hobbies, and future plans.

[0841] "Emotional state" refers to information about the user's psychological response, extracted from their facial expressions, voice, and other similar data.

[0842] A "generative model" is an algorithm that generates design proposals based on user attribute information and emotional state.

[0843] A "design proposal" is an initial plan for a house that takes into account the user's needs and feelings.

[0844] A "three-dimensional model" is a three-dimensional data representation that visually shows a design proposal in a virtual space.

[0845] A "virtual space" is a digital environment created by a computer system that is different from reality.

[0846] "Emotional feedback" refers to response information based on the user's emotional state.

[0847] "Emotion recognition technology" refers to technical methods for analyzing a user's emotional state.

[0848] "Real-time" refers to a time concept that describes the process by which a user's emotional state is analyzed and reflected immediately.

[0849] The system implementing this invention generates and presents individually optimized housing designs in a virtual space based on attribute information such as the user's lifestyle, family structure, hobbies, and future plans, as well as emotional states obtained from facial expressions and voice using emotion recognition technology. The server collects the user's attribute information and generates design proposals using a generation AI model. The generated design proposals are projected as three-dimensional models into a virtual space via a VR device on the terminal. The user experiences the design proposals in this virtual space, and their emotional state is analyzed in real time using emotion recognition technology. Based on the emotional feedback, the server modifies the design proposals and provides the user with optimized designs. This makes it possible to realize an experience that aligns with emotions and achieve highly satisfying housing designs.

[0850] Specifically, the device uses an emotion recognition API to analyze the user's emotional state in real time through its camera and microphone. The server uses a generative AI model to generate new design proposals based on the collected attribute information and emotional state. The regenerated design proposals are then converted into 3D models and presented to the user in a virtual space on the device. The software used includes, for example, 3D engines such as Unity and Unreal Engine, and cloud services such as Amazon Web Services and Google Cloud Platform.

[0851] For example, if a user expresses emotions such as "surprise" or "joy" while viewing the living room design during a virtual tour, the system will use this reaction to suggest ways to further enhance the living room design. An example of a prompt might be, "If the user is enjoying the living room, suggest ways to further enhance its design."

[0852] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0853] Step 1:

[0854] The server receives attribute information from the user. As input, it collects data such as lifestyle, family structure, hobbies, and future plans. This provides the basic information needed to generate design proposals optimized for the user.

[0855] Step 2:

[0856] The server inputs the collected attribute information into a generating AI model to generate design proposals. The generating AI model extracts the necessary conditions for the design from the attribute information and outputs a design proposal based on those conditions. This creates a personalized house design proposal for the user.

[0857] Step 3:

[0858] The terminal projects the generated design proposal as a three-dimensional model into the virtual space. Using the design proposal received from the server as input, it creates a visual model that the user can actually experience using a 3D engine. This allows the user to virtually tour the design proposal.

[0859] Step 4:

[0860] Users explore a virtual space through a VR device. This allows users to review design proposals and intuitively grasp the layout and arrangement of each room. As a result, users can develop genuine opinions and feelings about the design.

[0861] Step 5:

[0862] The device uses emotion recognition technology to analyze the user's facial expressions and voice, detecting their emotional state in real time. It acquires the user's camera video and audio data as input and processes it using an emotion recognition API. This allows it to acquire and analyze current emotional data based on the user's reactions.

[0863] Step 6:

[0864] The server modifies the design proposal based on the detected emotional state. The AI ​​model receives emotional data as input and recalculates the design proposal, outputting it as an output. This ensures that the design elements that should be emphasized are reflected according to the user's emotions.

[0865] Step 7:

[0866] The server then remodels the revised design proposal into a 3D model and transfers it to the terminal. The terminal projects the new design proposal into the virtual space. This allows the user to experience the latest design proposal, which has been adjusted based on emotions, once again.

[0867] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0868] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0869] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0870] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0871] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0872] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0873] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0874] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0875] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0876] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0877] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0878] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0879] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0880] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0881] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0882] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0883] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0884] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0885] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0886] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0887] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0888] The following is further disclosed regarding the embodiments described above.

[0889] (Claim 1)

[0890] A means for collecting user attribute information and generating design proposals based on a generative model,

[0891] A means of projecting the generated design proposal as a three-dimensional model into a virtual space,

[0892] A system that includes means for revising design proposals based on user feedback.

[0893] (Claim 2)

[0894] The system according to claim 1, which processes user feedback in real time and incorporates it into the design proposal.

[0895] (Claim 3)

[0896] The system according to claim 1, which generates and modifies design proposals based on laws and environmental conditions, taking into account market trends and regional characteristics.

[0897] "Example 1"

[0898] (Claim 1)

[0899] A means of collecting user attribute information through an input device and transmitting it as digital data,

[0900] A means of generating initial design proposals using a generative AI model to analyze user attribute information,

[0901] A means having a display device for converting the generated design proposal into a three-dimensional model and projecting it into a virtual space,

[0902] A means for users to provide feedback on design adjustments during a virtual tour,

[0903] A means of receiving user feedback and incorporating it into design proposals in real time,

[0904] Information processing means for modifying design proposals based on requirements, taking into account laws and environmental conditions,

[0905] A system that includes this.

[0906] (Claim 2)

[0907] The system according to claim 1, which receives user feedback via a receiving device and immediately incorporates it into the design proposal.

[0908] (Claim 3)

[0909] The system according to claim 1, which modifies the design proposal based on legal and environmental conditions, taking into account market trend information and regional characteristics information.

[0910] "Application Example 1"

[0911] (Claim 1)

[0912] A means for collecting user attribute information and generating design proposals based on a generative model,

[0913] A means of projecting the generated design proposal as a three-dimensional model into a virtual space,

[0914] A means of revising the design based on user feedback,

[0915] A means for users to move around within a 3D model during a virtual tour and grasp the details of specific parts,

[0916] A means of collecting real-time feedback while on the move and immediately revising the design proposal,

[0917] ...

[0918] A system that includes this.

[0919] (Claim 2)

[0920] The system according to claim 1, which processes user feedback in real time and incorporates it into the design proposal.

[0921] (Claim 3)

[0922] The system according to claim 1, which generates and modifies design proposals based on laws and environmental conditions, taking into account market trends and regional characteristics.

[0923] "Example 2 of combining an emotion engine"

[0924] (Claim 1)

[0925] A means for collecting user attribute information and generating design proposals based on a generative model,

[0926] A means of projecting the generated design proposal as a three-dimensional model into a virtual space,

[0927] A means of analyzing the user's facial expressions and voice to recognize their emotional state,

[0928] A means of optimizing design proposals based on recognized emotions,

[0929] Means for modifying the design proposal considering market trends, regional characteristics, laws and regulations, and environmental conditions,

[0930] A system that includes this.

[0931] (Claim 2)

[0932] The system according to claim 1, which processes user feedback in real time and incorporates it into the design proposal.

[0933] (Claim 3)

[0934] The system according to claim 1, which optimizes design proposals based on the user's emotional state.

[0935] "Application example 2 of combining emotional engines"

[0936] (Claim 1)

[0937] A means for collecting user attribute information and emotional state, and generating design proposals based on a generative model,

[0938] A method for projecting the generated design proposal as a three-dimensional model into a virtual space and modifying the design proposal based on emotional feedback,

[0939] A means of analyzing the emotions of users in a virtual space in real time using emotion recognition technology and reflecting them in design proposals,

[0940] Providing user-optimized design proposals and means to realize emotionally resonant experiences,

[0941] A system that includes this.

[0942] (Claim 2)

[0943] The system according to claim 1, which processes user emotional feedback in real time and optimizes design proposals based on the user's emotional state.

[0944] (Claim 3)

[0945] The system according to claim 1, which generates and modifies design proposals based on sentiment analysis data, taking into account market trends, regional characteristics, laws and regulations, and environmental conditions. [Explanation of Symbols]

[0946] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting user attribute information and generating design proposals based on a generative model, A means of projecting the generated design proposal as a three-dimensional model into a virtual space, A system that includes means for revising design proposals based on user feedback.

2. The system according to claim 1, which processes user feedback in real time and incorporates it into the design proposal.

3. The system according to claim 1, which generates and modifies design proposals based on laws and environmental conditions, taking into account market trends and regional characteristics.

Images