System

Abstract

To provide a system for generating a 3D model by inputting the feature of a house desired by a user, analyzing it by a server based on it, and using a generation AI.SOLUTION: A means for inputting a feature of a house desired by a user, a means for receiving and analyzing the input data of the user by a server, a means for generating a 3D model by using an AI generated based on the analysis data by the server, a means for transmitting the generated 3D model data to the user terminal by the server, a means for confirming and inputting a correction instruction by the user, a means for re-generating the 3D model by the server in response to the correction instruction, a means for final confirmation and finalizing the order by the user, a means for transmitting the order data to a construction company by the server and managing the project by the server, and a means for notifying the user terminal of the progress information of the project by the server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The traditional home ordering process makes it difficult to customize a home to meet the specific needs of the customer, which often leads to delivery delays and dissatisfaction after completion. Furthermore, the process from home design to construction is extremely time-consuming and costly, placing a great burden on customers. Another issue is the lengthening of mortgage terms. To solve these problems, a new method is needed to efficiently and quickly deliver the ideal home to customers. [Means for solving the problem]

[0005] This invention provides a system in which a user inputs the characteristics of a desired home, and a server analyzes the input and generates a 3D model using generative AI. The user checks the 3D model and inputs corrections as necessary, and the server regenerates it, ultimately confirming the order. This process makes it possible to provide the ideal home that reflects the user's wishes in a short period of time. Furthermore, the server sends the order data to the construction company, manages the project, and notifies the user's device of progress information, ensuring transparency and efficiency. This effectively solves the conventional problems.

[0006] "User" means an individual or entity that uses the system to order and configure a home.

[0007] A "terminal" is a device operated by a user, which executes the ordering app and receives user input and feedback.

[0008] The "server" is a central processing unit that receives and analyzes data sent by users and generates 3D models using generative AI.

[0009] "Generative AI" refers to algorithms or programs that use artificial intelligence techniques to automatically generate 3D models based on user requirements.

[0010] A "3D model" is digital data that represents the specific design of a house in three dimensions.

[0011] A "construction company" is a company or organization that carries out the construction of a house based on the user's order data.

[0012] The "ordering app" is application software that allows users to input the characteristics of a home and proceed through the ordering process.

[0013] "Project" refers to the series of tasks and their progress related to the design and construction of a home ordered by a user.

[0014] "Progress information" is data that indicates the progress of each stage of a construction project and is notified to the user.

[0015] "Modification instructions" are input data that indicate changes or improvements that the user wishes to make to the 3D model. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system for efficiently designing and quickly providing a home that a user desires. This system inputs the characteristics of the home that the user desires, analyzes that information, and generates a 3D model of the home. Details of the system and its embodiments are described below.

[0038] User requirements collection

[0039] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen, where they enter details such as their name, contact information, desired home style (e.g., natural modern, minimalist, classic, etc.), floor plan, budget, and materials used.

[0040] 3D model generation and modification

[0041] The device temporarily stores the information entered by the user in a local database and then sends it to the server. The server then analyzes the data and uses generative AI to generate an initial 3D model based on the user's requests.

[0042] The generated 3D model is sent to the device and displayed to the user. The user reviews the 3D model and indicates any necessary modifications. For example, they can input specific requests such as "I want to make the living space larger" or "I want to change the location of the kitchen." The device then sends the modifications back to the server, which then regenerates the 3D model based on those instructions. This modification process is repeated until the user is satisfied.

[0043] Final order confirmation

[0044] When the user is satisfied with the final model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[0045] Project management and progress notifications

[0046] Once the order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check the progress status through the app, such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color."

[0047] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[0048] Specific examples

[0049] Example 1:

[0050] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[0051] Example 2:

[0052] Another user requests a minimalist 2LDK layout. They set their budget and materials in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests a change in the location of the bedroom. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[0053] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The user downloads and installs the ordering app. The user launches the app and enters their user information (name, contact details).

[0057] Step 2:

[0058] The terminal displays a form for the user to input the characteristics of the home they desire (style, layout, budget, materials, etc.). The user enters this information.

[0059] Step 3:

[0060] The terminal temporarily stores the user's input data in a local database and then transmits it to the server.

[0061] Step 4:

[0062] The server receives and analyzes the user's input data. Based on the analysis results, it activates the generation AI to generate the initial 3D model.

[0063] Step 5:

[0064] The server sends the generated 3D model to the device, which displays the 3D model to the user and asks for confirmation.

[0065] Step 6:

[0066] The user checks the 3D model and inputs any modifications they would like to make, such as making the living space larger.

[0067] Step 7:

[0068] The device sends the user's correction instructions back to the server, which receives the instructions, uses the generation AI to correct the 3D model again, and sends it back to the device.

[0069] Step 8:

[0070] This process is repeated several times until a final model that satisfies the user is determined, and the user presses the "Confirm Order" button.

[0071] Step 9:

[0072] The terminal sends the final 3D model data and order information to the server, which then aggregates the order details and sends the order data to the construction company.

[0073] Step 10:

[0074] The server sends a notification of order confirmation to the terminal, which receives the notification and displays it to the user.

[0075] Step 11:

[0076] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[0077] Step 12:

[0078] The server periodically sends progress information to the terminal, which then notifies and displays the progress information to the user.

[0079] Step 13:

[0080] The user can check the progress and enter feedback as needed, for example, a request such as "I would like to change the color of the exterior walls."

[0081] Step 14:

[0082] The device sends the feedback to the server, which forwards it to the construction company.

[0083] Step 15:

[0084] The construction company makes corrections based on the feedback and performs a final check. The server notifies the user of completion and the date and time of handover.

[0085] Step 16:

[0086] The user makes a final check and the delivery is completed in a satisfactory manner.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] The traditional residential design process required a great deal of time and effort to meet the detailed needs of the user, making it difficult to design efficiently. Furthermore, there was a lack of collaboration and communication between the user and the construction company, making it difficult to make design changes or check progress. This could lead to lower customer satisfaction and project delays.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes: means for a user to input the characteristics of a home desired by the user; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model using a generative AI based on the analyzed data; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; means for the user to input order and correction data through an application; means for the server to operate the generative AI model using prompt sentences based on the user's input data; and means for the terminal to transmit order confirmation information from the user to the server. This enables home design that can quickly and efficiently meet the user's detailed needs, facilitating project progress management and communication with users.

[0092] A "user" is an individual or corporation that wishes to design and order a home.

[0093] An "ordering app" is a software application that allows users to enter the information necessary to design and order a home.

[0094] The "server" is a computer system that has the function of analyzing data collected from users, generating and modifying 3D models using generative AI, and finally transmitting order data to construction companies.

[0095] "Terminal" refers to a device that allows a user to input information via the ordering app, check the 3D model, and input correction instructions.

[0096] A "construction company" is a company or organization that actually constructs a house based on the order data sent from the server.

[0097] "Generative AI" is an artificial intelligence technology that automatically generates and modifies 3D models of homes based on user input data.

[0098] "3D Model" means a three-dimensional digital model of a house generated based on a user's requirements.

[0099] "Modification instructions" are input information for changes or adjustments that the user makes to the 3D model after checking it.

[0100] A "prompt" is a textual instruction used by the server to operate a generative AI model.

[0101] "Project progress information" is data that indicates the status and progress of housing projects currently underway by construction companies.

[0102] "Feedback" refers to additional requests or change requests made by users during the progress of a construction project.

[0103] "Confirmed order information" is the final confirmation data that is sent when the user is satisfied with the final model and officially orders the house.

[0104] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[0105] User requirements collection

[0106] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen where they enter details such as their name, contact information, desired home style (natural modern, minimalist, classic, etc.), floor plan, budget, and materials used. Information is entered using a device such as a smartphone or tablet.

[0107] 3D model generation and modification

[0108] The device temporarily stores the information entered by the user in a local database and then sends it to a server. The server used here is, for example, a cloud-based server system. The server that receives the data first analyzes the data and then uses a generative AI (such as OpenAI's DALL-E or Autodesk's Revit) to generate an initial 3D model. This 3D model is generated based on the user's requirements.

[0109] The generated 3D model is sent to the device and displayed to the user. The user checks the 3D model and, if any modifications are needed, inputs specific instructions. For example, instructions might be to "make the living space larger" or "change the location of the kitchen." The device then sends these instructions back to the server.

[0110] The server regenerates the 3D model based on the modification instructions, again using the generated AI model and prompts to make specific modifications to the model. The modified 3D model is then sent back to the user's device, and the process is repeated until the user is satisfied.

[0111] Final order confirmation

[0112] When the user is satisfied with the final 3D model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[0113] Project management and progress notifications

[0114] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check progress status such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color" through the app.

[0115] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[0116] Specific examples

[0117] Example 1:

[0118] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[0119] Example 2:

[0120] Another user requests a minimalist two-bedroom apartment, specifying their budget and materials to be used in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests changes to the bedroom location. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[0121] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] Step 1:

[0124] The user downloads and installs the ordering app. The user launches the app and enters details such as name, contact information, desired home style, layout, budget, and materials on the basic information input screen.

[0125] Input: User information (name, contact details, house style, layout, budget, materials used, etc.)

[0126] Output: User information entered

[0127] Specific actions: The user operates a smartphone or tablet, fills in the required information in the information input form, and presses the submit button.

[0128] Step 2:

[0129] The terminal temporarily stores the entered user information in a local database and then transmits the information to the server.

[0130] Input: User information (obtained from local database)

[0131] Output: Data sent to the server (user information)

[0132] Specific operation: The device sends user information to the server using an HTTP request.

[0133] Step 3:

[0134] The server analyzes the received user information and generates an initial 3D model using generative AI (e.g., OpenAI's DALL-E or Autodesk's Revit).

[0135] Input: User information (analyzed by the server)

[0136] Output: Initial 3D model data

[0137] Specific operation: The server parses the user information and asks the generation AI to generate a 3D model using a prompt, such as "Please generate a 3D model of a natural modern style 4LDK house."

[0138] Step 4:

[0139] The server sends the generated 3D model data to the terminal and displays it to the user.

[0140] Input: Initial 3D model data

[0141] Output: Data sent to the device (initial 3D model)

[0142] Specific operation: The server returns the generated 3D model data to the terminal, and the terminal displays the 3D model to the user.

[0143] Step 5:

[0144] The user checks the 3D model displayed on the device and inputs any necessary corrections, such as "make the living space larger" or "change the location of the kitchen."

[0145] Input: Initial 3D model, user instructions for modification

[0146] Output: Correction instruction data

[0147] Specific action: The user enters the specific changes in text format into the correction instruction input form of the app and presses the submit button.

[0148] Step 6:

[0149] The terminal transmits a correction instruction to the server.

[0150] Input: Correction instruction data (obtained from the terminal)

[0151] Output: Data to be sent to the server (correction instructions)

[0152] Specific operation: The device sends correction instructions to the server using an HTTP request.

[0153] Step 7:

[0154] The server regenerates the 3D model based on the received modification instructions, and again uses a generation AI (such as OpenAI's DALL-E or Autodesk's Revit) to modify the model using prompts.

[0155] Input: Correction instruction data

[0156] Output: Regenerated 3D model data

[0157] Specific operation: The server analyzes the modification instructions and sends a regeneration prompt to the generation AI, such as "Please regenerate a 3D model of a 4LDK natural modern style house with an expanded living space."

[0158] Step 8:

[0159] The server transmits the regenerated 3D model data to the terminal and displays it to the user.

[0160] Input: Regenerated 3D model data

[0161] Output: Data sent to the device (corrected 3D model)

[0162] Specific operation: The server sends the regenerated 3D model to the device, which then displays it to the user. This process is repeated until the user is satisfied.

[0163] Step 9:

[0164] If the user is satisfied with the final model, he or she presses the order confirmation button.

[0165] Input: Final 3D model, order confirmation instructions

[0166] Output: Confirmed order data

[0167] Specific operation: The user presses the order confirmation button, and the terminal sends the confirmation data to the server.

[0168] Step 10:

[0169] The terminal transmits the final 3D model data and order information to the server.

[0170] Input: Confirmed order data

[0171] Output: Data sent to the server (final 3D model data, order information)

[0172] What happens: The device sends the final 3D model and order data to the server using an HTTP request.

[0173] Step 11:

[0174] The server transmits the received order data to affiliated construction companies and manages the project.

[0175] Input: Order data

[0176] Output: Data to be sent to construction companies (order data)

[0177] Specific operation: The server sends the order data to the construction company and registers it in the project management system.

[0178] Step 12:

[0179] The server notifies the user's terminal of the progress information of the project received from the construction company.

[0180] Input: Progress information (received from construction company)

[0181] Output: Notification data (progress information) sent to the user's device

[0182] Specific operation: The server periodically checks progress information and sends notifications to the user's device at appropriate times.

[0183] This allows users to move through the home design process efficiently and quickly, facilitating overall communication and management right up to the final order.

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] There is a demand for efficient design of the homes users desire and a highly reproducible virtual viewing experience. However, conventional systems were unable to quickly generate and modify 3D models that matched the user's wishes, and lacked a means to check the progress in real time. In addition, users could not actually view the home before it was completed, which could lead to low satisfaction with the final home design.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes: means for a user to input the characteristics of a desired home; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model based on the analysis data using a generation AI; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; and means for displaying the 3D model in a VR space on the user terminal, allowing the user to virtually tour the home. This allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[0189] A "user terminal" is a device used by a user to input the characteristics of a house, check the 3D model, input correction instructions, and take a virtual tour.

[0190] The "server" is a device or system that receives and analyzes data sent from user devices and generates 3D models using generative AI. It also manages project progress information and notifies user devices.

[0191] "Generative AI" is an artificial intelligence model that automatically generates a 3D model based on the characteristic data of a house entered by the user.

[0192] A "3D model" is a digital model that represents the shape, layout, style, etc. of the home desired by the user in three dimensions.

[0193] "VR space" is a virtual space that uses virtual reality technology to allow users to tour homes in a virtual environment.

[0194] "Modification instructions" are input information that a user can use to request changes or improvements to specific parts of a 3D model after checking it.

[0195] A "virtual tour" is a virtual home tour method that allows users to use a VR space to actually walk around the generated 3D model and experience it for themselves.

[0196] "Progress information" is data that indicates the progress of construction and important events in a user's housing project.

[0197] MODE FOR CARRYING OUT THE INVENTION

[0198] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[0199] User requirements collection

[0200] First, users install a dedicated application on their smartphone. When they launch the application, a screen appears where they can enter details such as the desired style of the home (e.g., natural modern, classic, minimalist, etc.), floor plan, budget, and materials used. This data is temporarily stored on the user's device and then sent to the server.

[0201] 3D model generation and modification

[0202] The server receives and analyzes the data sent by the user. Based on the analysis results, the server uses a generative AI model to generate an initial 3D model. After generation, this 3D model is sent to the user's device and displayed in the application. The user reviews the 3D model and, if any modifications are necessary, enters specific instructions (e.g., "make the living room bigger"). Based on the modification instructions sent back to the server, the server regenerates a new 3D model and sends it back to the user's device. This process is repeated until the user is satisfied.

[0203] Virtual Tour

[0204] The user's device can display the generated 3D model in a VR space. The user can attach a smartphone to a VR headset and take a virtual tour. This allows the user to experience the interior of the designed home in real time and check the details.

[0205] Final order confirmation and construction management

[0206] Once the user is satisfied with the final model and places an order, the order data is sent to the server. The server aggregates the order information and sends it to the partner construction company. This process starts the construction project. The server periodically receives progress information from the construction company and notifies the user's device in real time. The user can check the project's progress (e.g., "foundation work completed") at any time through the application.

[0207] Specific examples

[0208] A user uses a smartphone app to input desired housing information. For example, they enter information such as "Style: Natural Modern," "Layout: 4LDK," "Budget: 50 million yen," and "Materials used: Wood, glass, concrete." The server receives this information and generates a 3D model using a generative AI model. The prompt statement "Please generate a 3D model of a 4LDK natural modern home based on the following information: Style: Natural Modern, Layout: 4LDK, Budget: 50 million yen, Materials used: Wood, glass, concrete" is passed to the generative AI model.

[0209] The user can view the generated 3D model in VR space, and if they feel there is a problem with the living space, they can input a correction instruction such as "make the living space larger." The server receives this correction instruction and updates the 3D model again using the generated AI model. This process is repeated until a model that satisfies the user is finally completed. Once the user confirms the final order, the construction project begins, and progress information is notified to the user in real time.

[0210] This system allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The user launches the smartphone app and inputs the desired housing characteristics (e.g., style, layout, budget, materials used, etc.). The input data is initially stored on the user's device and later sent to the server.

[0214] Input: House characteristic data entered by the user.

[0215] Output: House feature data temporarily stored on the user's device.

[0216] Step 2:

[0217] The device sends the user's input data to the server, which then analyzes the data. This analysis includes data validation and missing value completion.

[0218] Input: House characteristic data sent from the user terminal.

[0219] Output: Analyzed housing data stored on the server.

[0220] Step 3:

[0221] The server generates an initial 3D model using a generative AI model based on the analyzed data. A prompt is generated and input into the generative AI to generate the 3D model.

[0222] Input: Parsed housing data, prompt for the generative AI model.

[0223] Output: The generated 3D model data.

[0224] Step 4:

[0225] The server sends the generated 3D model data to the user's device, where the 3D model is displayed and the user can confirm the model.

[0226] Input: Generated 3D model data.

[0227] Output: The 3D model displayed on the user's device.

[0228] Step 5:

[0229] The user checks the 3D model and inputs correction instructions as necessary. The correction instructions are entered into the terminal as specific requests (e.g., "Make the living room bigger") and sent to the server.

[0230] Input: Correction instructions entered by the user.

[0231] Output: Correction instruction data sent to the server.

[0232] Step 6:

[0233] The server regenerates the 3D model based on the received correction instructions, and then passes the prompt sentence to the generative AI model again to generate an updated 3D model.

[0234] Input: User correction instruction data, update prompt sentence to the generative AI model.

[0235] Output: Updated 3D model data.

[0236] Step 7:

[0237] The server then sends the updated 3D model back to the user's device, where the user reviews the new 3D model and repeats this process until the modifications are complete.

[0238] Input: Updated 3D model data.

[0239] Output: The updated 3D model displayed on the user's device.

[0240] Step 8:

[0241] When the user finally completes a model that satisfies them, they press the order confirmation button, and the terminal sends the confirmed order data to the server.

[0242] Input: Final confirmed 3D model, order confirmation information.

[0243] Output: Order confirmation data sent to the server.

[0244] Step 9:

[0245] The server sends the order information to the construction company based on the received order confirmation data. It also starts managing the construction project, periodically receives progress information from the construction company, and notifies the user terminal.

[0246] Input: Order confirmation data, project progress information.

[0247] Output: Order information sent to the construction company, progress information notified to the user terminal.

[0248] Step 10:

[0249] The 3D model is displayed in a VR space so that users can take a virtual tour through their devices. Users can connect their smartphone to a VR headset and enjoy the virtual tour experience.

[0250] Input: 3D model data to be displayed in VR space.

[0251] Output: 3D model displayed in a VR headset for virtual tours.

[0252] Through these steps, users can design their desired home in detail, check, modify, and finalize the design in virtual space, and keep track of the project's progress in real time.

[0253] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0254] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[0255] User requirement collection and emotion recognition

[0256] A user downloads and installs the ordering app. When the user launches the app, a screen for entering basic information appears. The ordering app has the ability to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions during the process.

[0257] Generating 3D models and reflecting emotion data

[0258] The device sends the user's input data and the recognized emotion data to the server. The server then first analyzes the characteristic data of the user's desired home, and then analyzes the emotion data. Based on the analysis results, the generative AI then activates and generates an initial 3D model that takes the user's emotion data into account. This process makes it possible, for example, to design a home that incorporates elements that the user feels positive about.

[0259] Emotional reflection of 3D model checking and correction

[0260] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs corrections they would like to make to the model, their emotions are also collected at the same time. For example, along with instructions such as "I would like the living space to be a little larger," the user's emotions at that time (dissatisfaction, expectations, etc.) are sent.

[0261] The device sends these correction instructions and emotion data to the server, which then uses the generative AI to correct the 3D model. The emotion data is used to predict the importance of the user's request and makes optimal corrections based on that.

[0262] Final order confirmation

[0263] When the user is satisfied with the final model, they press the order confirmation button. The device then sends the final 3D model data, order information, and emotion data to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is then sent to the device as a notification to the user.

[0264] Emotional reflection in project management and progress notifications

[0265] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at the set timing. The emotion engine also operates when notifications are sent, capturing the user's emotions when providing feedback and sending them to the server.

[0266] Specific examples

[0267] Example 1:

[0268] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[0269] Example 2:

[0270] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[0271] In this way, by incorporating an emotion engine, the system of the present invention can design a house that reflects the user's feelings, thereby achieving higher customer satisfaction.

[0272] The processing flow will be explained below.

[0273] Step 1:

[0274] A user downloads and installs the ordering app. When the user launches the app, a screen appears where the user can enter their information (name, contact details).

[0275] Step 2:

[0276] In addition to a screen for entering basic information, the device also activates an emotion engine that captures the user's facial expressions and voice. As the user enters information, the emotion engine captures the user's emotional data in real time.

[0277] Step 3:

[0278] The user inputs the home's characteristics (style, layout, budget, materials used, etc.) into the ordering app. The emotion engine captures the user's emotions while they are typing.

[0279] Step 4:

[0280] The device temporarily stores the user's input data and emotion data in a local database and then transmits it to the server.

[0281] Step 5:

[0282] The server receives and analyzes the user's input data, then analyzes the emotion data. Based on these analysis results, a generative AI is activated to generate an initial 3D model that takes the user's emotions into account.

[0283] Step 6:

[0284] The server sends the generated 3D model and the emotion analysis results to the device, which then displays the 3D model to the user and asks for their confirmation.

[0285] Step 7:

[0286] The user checks the 3D model and inputs the parts they want to modify. At the same time, the emotion engine captures the user's emotions and sends them, along with the modification instructions, from the device to the server.

[0287] Step 8:

[0288] The server receives the modification instructions and emotion data, and again modifies the 3D model using the generative AI. At this time, the model is adjusted with an emphasis on the user's emotion data. For example, it focuses on improving areas that the user expressed dissatisfaction with.

[0289] Step 9:

[0290] The server resends the modified 3D model to the device, which displays it to the user and asks for confirmation again. This process is repeated until the user is satisfied.

[0291] Step 10:

[0292] If the user is satisfied with the final model, they press the "Confirm Order" button, and the device sends the final 3D model data, order information, and emotion data to the server.

[0293] Step 11:

[0294] The server collects the order details and sends the order data to the affiliated construction company. This confirmation information is then sent to the terminal and notified to the user.

[0295] Step 12:

[0296] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[0297] Step 13:

[0298] The server periodically sends progress information to the device. The device notifies the user of the progress and displays it. The emotion engine also works when notifying the user, capturing the emotions expressed by the user regarding the progress.

[0299] Step 14:

[0300] Users can check the progress and enter feedback as needed. For example, requests such as "I want to change the color of the exterior walls" are captured along with their emotions at the time.

[0301] Step 15:

[0302] The device sends the user's feedback and emotional data to a server, which analyzes the feedback and forwards it to the construction company.

[0303] Step 16:

[0304] The construction company makes corrections based on the feedback and performs a final check. The server sends a completion notification to the terminal and notifies the user of the date and time of handover. The user makes a final check and the handover is completed in a satisfactory manner.

[0305] Example 2

[0306] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0307] Conventional home design systems can collect specific user requirements, but do not consider the emotions felt by the user during the design process. As a result, the user's emotions and satisfaction are not reflected in the design process, which can lead to dissatisfaction with the final home design. In particular, there is a need for a system that can accurately reflect the user's emotions when making changes to the layout or style of the home.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0309] In this invention, the server includes: a means for a user to input the characteristics of a home desired by the user; a means for the terminal to capture the user's facial expressions and voice in real time and recognize emotions; a means for the server to receive and analyze the user's input data and emotional data; a means for the server to generate a 3D model using a generation AI based on the analyzed data; a means for the server to send the generated 3D model data to the terminal; a means for the user to check the 3D model and input correction instructions; a means for the terminal to recapture the user's facial expressions and voice and recognize emotions; a means for the server to receive the correction instructions and emotional data and regenerate the 3D model; a means for the user to make a final confirmation and confirm the order; a means for the server to send the order data to a construction company and manage the project; and a means for the server to notify the terminal of project progress information. This makes it possible to reflect the user's emotions in the home design process and provide a final home design that will provide a high level of satisfaction.

[0310] "User" refers to an individual or corporation that uses the system to design a desired home.

[0311] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[0312] "Facial and audio capture" refers to the process by which the device records the user's facial movements and voice in real time.

[0313] "Emotion recognition" refers to analyzing a user's emotional state (e.g., joy, sadness, anticipation, dissatisfaction) from captured facial and voice data.

[0314] "Server" refers to a central computing device that processes and manages data for the entire system.

[0315] "Input data" refers to information including the characteristics and requirements of the home desired by the user.

[0316] "Emotion data" refers to information about emotions recognized from the user's facial expressions and voice.

[0317] "Analyzing" refers to the server processing the received data with an analysis algorithm.

[0318] "Generative AI" refers to a system that uses artificial intelligence technology to generate a 3D model of a house based on user requirements and emotional data.

[0319] "3D model" refers to three-dimensional design data for a house that is generated based on the user's wishes and emotional data.

[0320] "Sending" refers to the process by which the server transfers the generated data to the terminal.

[0321] "Modification instructions" refer to requests for changes or adjustments made by the user to the 3D model.

[0322] "Regenerate" refers to the server recreating the 3D model based on the correction instructions and emotion data.

[0323] "Final confirmation" refers to when the user is satisfied with the final 3D model and confirms the order.

[0324] "Order Data" means all order information included with the final confirmed 3D model.

[0325] "Construction company" refers to the corporation that actually constructs the housing ordered.

[0326] "Managing the project" refers to the server monitoring and managing the progress of the entire home building process.

[0327] "Progress Information" means data relating to the progress of a construction project.

[0328] "Notifying" refers to the process by which the server communicates progress and other important information to the device.

[0329] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[0330] User requirement collection and emotion recognition

[0331] A user downloads and installs a home design app on their smartphone or tablet. When the app is launched, a screen for entering basic information appears. The app is equipped with a function to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions in real time. Specifically, the app uses the smartphone's camera and microphone to acquire emotion data using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[0332] Generating 3D models and reflecting emotion data

[0333] The device sends the user's input data and emotion data to the server. Upon receiving this data, the server first analyzes the user's request data, and then analyzes the emotion data. Based on the analysis results, the server activates the generative AI to generate an initial 3D model that reflects the user's emotion data. The server then sends the generated 3D model data to the device, and uses an interactive viewer (e.g., Three.js) to display the 3D model on the smartphone screen.

[0334] Emotional reflection of 3D model checking and correction

[0335] When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs the parts of the model they want to modify, their emotions are also collected at the same time. Specifically, the camera and microphone are used again to collect the user's emotional data. The device sends these modification instructions and emotional data to the server, and the server again uses the generative AI to modify the 3D model. The emotional data is used to determine the importance of the user's request, and the optimal modifications are made based on that.

[0336] Final order confirmation and project management

[0337] When the user is satisfied with the final model, they press the order confirmation button. The device sends the final 3D model data, order information, and emotion data to the server. The server consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user. Once the project begins, the server manages the progress of the construction project. The construction company periodically reports progress information to the server, and the server receives this progress information and notifies the device at the set time. The emotion engine also runs when notifying, capturing the emotion of the user's feedback and sending it to the server.

[0338] Examples and thought-provoking prompts

[0339] Example 1

[0340] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[0341] Prompt Sentence Examples

[0342] If a user looking for a natural modern 4LDK home expresses dissatisfaction with the size of the living space, please suggest how to improve it based on emotional data.

[0343] Example 2

[0344] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[0345] Prompt Sentence Examples

[0346] If a user with a budget constraint requests to change the location of the bedroom in a minimalist 2LDK apartment, please suggest the optimal layout based on the sentiment data.

[0347] In this way, by incorporating an emotion engine, the system of the present invention can design a home that reflects the user's feelings, achieving higher customer satisfaction. The hardware used to implement this system includes a smartphone and a server, and specific software includes facial recognition software (e.g., OpenFace), voice analysis software (e.g., IBM Watson), and an interactive viewer (e.g., Three.js).

[0348] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0349] Step 1:

[0350] A user downloads and installs a home design app.

[0351] What it does: Download and install the app from the app store on your smartphone or tablet.

[0352] Input: None

[0353] Output: Device with home design app installed

[0354] Step 2:

[0355] The user launches the app and is presented with a screen for entering basic information.

[0356] Specific operation: When the user taps the app icon to launch the app, a screen for entering basic information appears on the device.

[0357] Input: None

[0358] Output: Basic information input screen

[0359] Step 3:

[0360] Users enter details such as their name, contact information, desired home style and layout, budget, and materials used.

[0361] Specific behavior: The user enters the required information into the form and proceeds to the next screen.

[0362] Input: User-entered house characteristics data

[0363] Output: The entered house feature data is saved on the device.

[0364] Step 4:

[0365] The device captures the user's facial expressions and voice in real time and recognizes their emotions.

[0366] Specific operation: The smartphone's camera and microphone are activated, and emotion data is obtained using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[0367] Input: User's facial expression and voice data

[0368] Output: Emotion data

[0369] Step 5:

[0370] The terminal transmits the user's input data and emotion data to the server.

[0371] Specific behavior: The input data and captured emotion data are sent to the server using the HTTPS protocol.

[0372] Input: House feature data, emotion data

[0373] Output: Data sent to the server

[0374] Step 6:

[0375] The server analyzes the received data and generates an initial 3D model using a generative AI model.

[0376] Specific operation: The server analyzes the received data and obtains the analysis results using a deep learning model (e.g., TensorFlow). Based on that data, it uses generative AI (e.g., Unity3D or Blender) to render a 3D model.

[0377] Input: House feature data, emotion data

[0378] Output: Initial 3D model

[0379] Step 7:

[0380] The server sends the generated 3D model data to the terminal.

[0381] Specific operation: The generated 3D model data is encoded and sent to the terminal using the HTTPS protocol.

[0382] Input: Initial 3D model

[0383] Output: 3D model sent to device

[0384] Step 8:

[0385] The device displays the 3D model to the user.

[0386] Specific operation: Use an interactive viewer (e.g., Three.js) to display a 3D model on the smartphone screen and allow the user to check it.

[0387] Input: 3D model data

[0388] Output: A displayed 3D model

[0389] Step 9:

[0390] The user checks the 3D model and inputs correction instructions.

[0391] Specific operation: The user adjusts the model using the touch interface of the smartphone.

[0392] Input: User correction instructions

[0393] Output: Correction instruction data

[0394] Step 10:

[0395] The device recaptures the user's facial expressions and voice to recognize emotions.

[0396] Specific operation: Again, the camera and microphone are used to collect user emotion data.

[0397] Input: User's facial expression and voice data

[0398] Output: Emotion data at the time of correction

[0399] Step 11:

[0400] The device sends correction instructions and emotion data to the server.

[0401] Specific operation: Input data and emotion data are sent to the server using the HTTPS protocol.

[0402] Input: Correction instruction data, emotion data when correcting

[0403] Output: Correction data and emotion data sent to the server

[0404] Step 12:

[0405] The server regenerates the 3D model based on the correction instructions and emotion data.

[0406] Specific operation: The server integrates the correction instructions and emotion data, and again uses the generative AI to generate a corrected 3D model.

[0407] Input: Correction instruction data, emotion data when correcting

[0408] Output: Modified 3D model

[0409] Step 13:

[0410] If the user is satisfied with the final model, he or she presses the order confirmation button.

[0411] Specific operation: The user presses the order confirmation button on the smartphone.

[0412] Input: User order confirmation operation

[0413] Output: Order confirmation data

[0414] Step 14:

[0415] The terminal sends the final 3D model data, order information, and emotion data to the server.

[0416] Specific behavior: The final data is sent to the server using the HTTPS protocol.

[0417] Input: Final 3D model data, order information, emotion data

[0418] Output: Final data sent to the server

[0419] Step 15:

[0420] The server collects the order information and sends the order data to the affiliated construction company.

[0421] Specific operation: The server sends the order data to the construction company via email or API.

[0422] Input: Order details data

[0423] Output: Order data sent to construction company

[0424] Step 16:

[0425] The server notifies the user of the order confirmation information.

[0426] Specific operation: The server sends a push notification to the device to notify the user that the order has been confirmed.

[0427] Input: Order confirmation information

[0428] Output: Push notification to the user

[0429] Step 17:

[0430] The server manages the progress of the construction project.

[0431] What it does: The server uses project management software (e.g. Jira) to track progress.

[0432] Input: Project Management Information

[0433] Output: Managed progress data

[0434] Step 18:

[0435] The construction company reports progress information to the server.

[0436] What happens: A construction company uses a progress reporting tool (e.g., Microsoft Project) to send progress data to a server.

[0437] Input: Progress information

[0438] Output: Progress data sent to the server

[0439] Step 19:

[0440] The server notifies the user of progress information.

[0441] Specific operation: The server sends a push notification to the device at the set timing.

[0442] Input: Progress information

[0443] Output: Progress notification to user

[0444] Step 20:

[0445] The device will display a notification and the emotion engine will capture the emotion during the feedback.

[0446] Specific behavior: When the user confirms the notification, the device again uses the camera and microphone to capture emotion data.

[0447] Input: Emotional data as user feedback

[0448] Output: Feedback emotion data sent to the server

[0449] (Application example 2)

[0450] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0451] The current home design process rarely takes into account user emotions, making it difficult to increase true user satisfaction. Virtual stores also lack personalized responses based on user emotions, creating a need for improved user experience. Conventional technologies lack the means to analyze user emotions in real time and customize products and services based on those emotions, making user satisfaction a challenge.

[0452] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input the characteristics of the house desired by the user; means for the terminal to capture the user's facial expressions and voice and acquire emotional data; means for the server to receive and analyze the user's input data and emotional data; means for the server to generate a 3D model based on the analyzed data and taking the emotional data into consideration using a generation AI; means for the server to send the generated 3D model data to the user terminal; means for the user to check the 3D model and input correction instructions; means for the terminal to re-capture the user's facial expressions and voice and acquire emotional data; means for the server to receive the correction instructions and emotional data and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to send the order data to the construction company and manage the project; and means for the server to notify the user terminal of project progress information. This makes it possible to provide more personalized products and services that reflect the user's emotions in real time.

[0453] The "means for the user to input the characteristics of the home desired" is a device or system that provides an interface for the user to specifically input the requirements and desires regarding the ideal home.

[0454] "Means for the device to capture the user's facial expressions and voice and obtain emotional data" refers to technology that captures the user's facial expressions and voice in real time, analyzes them, and obtains the user's emotions as data.

[0455] The "means for the server to receive and analyze the input data and emotion data of the user" refers to a device or system that receives data related to housing requirements and emotions sent from the user and performs an analysis process.

[0456] "Means for the server to use generative AI based on the analyzed data to generate a 3D model that also takes into account emotional data" refers to a technology that uses generative AI to create a 3D model that reflects the user's emotions based on analyzed data.

[0457] "Means by which the server transmits the generated 3D model data to the user terminal" refers to a mechanism for transferring the generated 3D model data to the terminal operated by the user.

[0458] "Means for the user to check the 3D model and input correction instructions" refers to a device or system that displays the 3D model to the user and provides an interface that allows the user to check it and then input correction instructions.

[0459] "Means for the device to recapture the user's facial expressions and voice and obtain emotional data" refers to technology that recaptures the user's facial expressions and voice and digitizes their emotions when checking and correcting the 3D model.

[0460] "Means for the server to receive correction instructions and emotion data and regenerate the 3D model" refers to technology that regenerates the 3D model based on correction instructions sent by the user and re-acquired emotion data.

[0461] "Means for the user to make final confirmation and confirm the order" refers to the procedures and devices by which the user can finally confirm the 3D model and officially confirm the order after being satisfied with it.

[0462] The "means for the server to transmit order data to the construction company and manage the project" is a system for transmitting confirmed order data to the construction company and managing the progress of the project.

[0463] "Means for the server to notify the user terminal of project progress information" refers to a technology that periodically notifies the terminal operated by the user of the progress status of the project.

[0464] The present invention is a system that recognizes a user's emotions and reflects them in the home design process, enabling more satisfying home designs. It can also be applied as a system for suggesting products in a virtual store based on the user's emotions. Detailed embodiments of the system are described below.

[0465] System Program

[0466] This system recognizes the user's emotions and analyzes their emotional data to suggest the most suitable 3D models and products for the user. This process includes the following elements:

[0467] 1. Obtaining user input and emotion data:

[0468] Users download a dedicated application and input the desired characteristics for their home design. The camera and microphone on their device (such as a smartphone or tablet) capture the user's facial expressions and voice in real time to obtain emotional data. The DeepFace library is used to analyze facial expressions and recognize key emotions (e.g., "happiness," "sadness," "surprise," etc.).

[0469] 2. Data Receipt and Analysis:

[0470] The input data and emotion data acquired by the device are sent to a server, where they are simultaneously analyzed and home design data that reflects the user's emotions is generated. This analysis is performed using machine learning models and generative AI.

[0471] 3. Generate 3D model:

[0472] Using generative AI, an initial 3D model is generated that takes into account the user's emotional data. This generation takes into account the user's desired home characteristics data and the results of emotional analysis.

[0473] 4. Check and correct the 3D model:

[0474] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model and inputs correction instructions or desired changes, the device again captures facial expressions and voice to obtain emotional data. This allows the device to analyze the importance of the correction instructions and the intensity of the emotion, allowing for more appropriate corrections.

[0475] 5. Final model confirmation and project management:

[0476] Once the user is satisfied with the final 3D model, they confirm their order. The server sends the order data to the construction company and manages the project progress. Project progress information is periodically sent to the user's device, and emotional data is also collected during the project.

[0477] Hardware and software used

[0478] Hardware:

[0479] Smartphone, tablet, or PC camera

[0480] microphone

[0481] software:

[0482] OpenCV

[0483] DeepFace

[0484] Hugging Face transformers library

[0485] Specific examples

[0486] 1. User's house design:

[0487] A user looking for a natural modern-style home inputs their information into the application. As they input, the camera and microphone capture their facial expressions and voice, and the emotion engine recognizes that the user is expressing strong expectations for the living space. Based on this, the generative AI creates an initial 3D model and presents it to the user.

[0488] Example prompt sentence:

[0489] "Users are expressing joy. Please suggest appropriate living space layouts."

[0490] 2. Product suggestions in virtual stores:

[0491] The facial expressions and voices of users shopping using a virtual store application are captured, and if the user expresses feelings of fatigue, the generative AI will suggest relaxing products (e.g., aroma candles).

[0492] Example prompt sentence:

[0493] "The user is showing signs of fatigue. Please suggest appropriate relaxation items."

[0494] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0495] Step 1:

[0496] A user downloads and installs a home design app. When the app is launched, a screen for entering basic information appears. While the user enters the desired home characteristics (style, layout, budget, materials used, etc.), the device's (smartphone or tablet) camera and microphone capture the user's facial expressions and voice in real time, obtaining emotion data. This data is obtained using the OpenCV and DeepFace libraries. The input is the user's information, facial expressions, and voice data, and the output is emotion data.

[0497] Step 2:

[0498] The input data (house features) and emotion data acquired by the device are sent to the server. The server receives the sent data and performs an analysis process. The server analyzes the house data entered by the user and the captured emotion data to identify the house features desired by the user. The input is the house features and emotion data, and the output is the analyzed feature data.

[0499] Step 3:

[0500] The server uses a generative AI based on the analyzed data to generate an initial 3D model that also takes into account emotional data. The Hugging Face generative AI model is used for this generation. The analysis results and a prompt are input into the generative AI model, which outputs a 3D model that reflects the user's emotions. The input is the analyzed feature data, emotional data, and the prompt "Please propose the layout of the living space that the user desires," and the output is the initial 3D model.

[0501] Step 4:

[0502] The server sends the generated 3D model data to the user's device. The user checks the 3D model on the device and inputs any complaints or corrections. At this time, the device again captures the user's facial expressions and voice, and obtains emotional data accompanying the corrections. The input is the 3D model data, and the output is the corrections and emotional data.

[0503] Step 5:

[0504] The device sends the correction instructions and emotion data to the server. The server receives them and regenerates the 3D model using the analysis and generative AI. The input is the correction instructions, emotion data, and the prompt "Please suggest the optimal layout of the living space based on the user's correction instructions." The output is the corrected 3D model.

[0505] Step 6:

[0506] Once the user has finalized the 3D model that satisfies them, they confirm their order. The terminal sends the final model data and order information to the server. The input is the confirmed 3D model and order information, and the output is the final order data.

[0507] Step 7:

[0508] The server sends order data to the construction company and manages the progress of the project. Progress information on ongoing projects is periodically reported to the server, which then notifies the user's device. When notifying the user, the device also captures the user's facial expression and collects emotional data. The input is progress information from the construction company, and the output is progress notification and emotional data to the user.

[0509] This series of processes enables the design of homes that reflect the user's emotions and provide a higher level of satisfaction. Furthermore, emotion-based product suggestions can be made as needed.

[0510] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0511] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0512] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0513] [Second embodiment]

[0514] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0515] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0516] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0517] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0518] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0519] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0520] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0521] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0522] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0523] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0524] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0525] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0526] The present invention is a system for efficiently designing and quickly providing a home that a user desires. This system inputs the characteristics of the home that the user desires, analyzes that information, and generates a 3D model of the home. Details of the system and its embodiments are described below.

[0527] User requirements collection

[0528] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen, where they enter details such as their name, contact information, desired home style (e.g., natural modern, minimalist, classic, etc.), floor plan, budget, and materials used.

[0529] 3D model generation and modification

[0530] The device temporarily stores the information entered by the user in a local database and then sends it to the server. The server then analyzes the data and uses generative AI to generate an initial 3D model based on the user's requests.

[0531] The generated 3D model is sent to the device and displayed to the user. The user reviews the 3D model and indicates any necessary modifications. For example, they can input specific requests such as "I want to make the living space larger" or "I want to change the location of the kitchen." The device then sends the modifications back to the server, which then regenerates the 3D model based on those instructions. This modification process is repeated until the user is satisfied.

[0532] Final order confirmation

[0533] When the user is satisfied with the final model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[0534] Project management and progress notifications

[0535] Once the order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check the progress status through the app, such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color."

[0536] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[0537] Specific examples

[0538] Example 1:

[0539] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[0540] Example 2:

[0541] Another user requests a minimalist 2LDK layout. They set their budget and materials in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests a change in the location of the bedroom. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[0542] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[0543] The processing flow will be explained below.

[0544] Step 1:

[0545] The user downloads and installs the ordering app. The user launches the app and enters their user information (name, contact details).

[0546] Step 2:

[0547] The terminal displays a form for the user to input the characteristics of the home they desire (style, layout, budget, materials, etc.). The user enters this information.

[0548] Step 3:

[0549] The terminal temporarily stores the user's input data in a local database and then transmits it to the server.

[0550] Step 4:

[0551] The server receives and analyzes the user's input data. Based on the analysis results, it activates the generation AI to generate the initial 3D model.

[0552] Step 5:

[0553] The server sends the generated 3D model to the device, which displays the 3D model to the user and asks for confirmation.

[0554] Step 6:

[0555] The user checks the 3D model and inputs any modifications they would like to make, such as making the living space larger.

[0556] Step 7:

[0557] The device sends the user's correction instructions back to the server, which receives the instructions, uses the generation AI to correct the 3D model again, and sends it back to the device.

[0558] Step 8:

[0559] This process is repeated several times until a final model that satisfies the user is determined, and the user presses the "Confirm Order" button.

[0560] Step 9:

[0561] The terminal sends the final 3D model data and order information to the server, which then aggregates the order details and sends the order data to the construction company.

[0562] Step 10:

[0563] The server sends a notification of order confirmation to the terminal, which receives the notification and displays it to the user.

[0564] Step 11:

[0565] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[0566] Step 12:

[0567] The server periodically sends progress information to the terminal, which then notifies and displays the progress information to the user.

[0568] Step 13:

[0569] The user can check the progress and enter feedback as needed, for example, a request such as "I would like to change the color of the exterior walls."

[0570] Step 14:

[0571] The device sends the feedback to the server, which forwards it to the construction company.

[0572] Step 15:

[0573] The construction company makes corrections based on the feedback and performs a final check. The server notifies the user of completion and the date and time of handover.

[0574] Step 16:

[0575] The user makes a final check and the delivery is completed in a satisfactory manner.

[0576] Example 1

[0577] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0578] The traditional residential design process required a great deal of time and effort to meet the detailed needs of the user, making it difficult to design efficiently. Furthermore, there was a lack of collaboration and communication between the user and the construction company, making it difficult to make design changes or check progress. This could lead to lower customer satisfaction and project delays.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0580] In this invention, the server includes: means for a user to input the characteristics of a home desired by the user; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model using a generative AI based on the analyzed data; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; means for the user to input order and correction data through an application; means for the server to operate the generative AI model using prompt sentences based on the user's input data; and means for the terminal to transmit order confirmation information from the user to the server. This enables home design that can quickly and efficiently meet the user's detailed needs, facilitating project progress management and communication with users.

[0581] A "user" is an individual or corporation that wishes to design and order a home.

[0582] An "ordering app" is a software application that allows users to enter the information necessary to design and order a home.

[0583] The "server" is a computer system that has the function of analyzing data collected from users, generating and modifying 3D models using generative AI, and finally transmitting order data to construction companies.

[0584] "Terminal" refers to a device that allows a user to input information via the ordering app, check the 3D model, and input correction instructions.

[0585] A "construction company" is a company or organization that actually constructs a house based on the order data sent from the server.

[0586] "Generative AI" is an artificial intelligence technology that automatically generates and modifies 3D models of homes based on user input data.

[0587] "3D Model" means a three-dimensional digital model of a house generated based on a user's requirements.

[0588] "Modification instructions" are input information for changes or adjustments that the user makes to the 3D model after checking it.

[0589] A "prompt" is a textual instruction used by the server to operate a generative AI model.

[0590] "Project progress information" is data that indicates the status and progress of housing projects currently underway by construction companies.

[0591] "Feedback" refers to additional requests or change requests made by users during the progress of a construction project.

[0592] "Confirmed order information" is the final confirmation data that is sent when the user is satisfied with the final model and officially orders the house.

[0593] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[0594] User requirements collection

[0595] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen where they enter details such as their name, contact information, desired home style (natural modern, minimalist, classic, etc.), floor plan, budget, and materials used. Information is entered using a device such as a smartphone or tablet.

[0596] 3D model generation and modification

[0597] The device temporarily stores the information entered by the user in a local database and then sends it to a server. The server used here is, for example, a cloud-based server system. The server that receives the data first analyzes the data and then uses a generative AI (such as OpenAI's DALL-E or Autodesk's Revit) to generate an initial 3D model. This 3D model is generated based on the user's requirements.

[0598] The generated 3D model is sent to the device and displayed to the user. The user checks the 3D model and, if any modifications are needed, inputs specific instructions. For example, instructions might be to "make the living space larger" or "change the location of the kitchen." The device then sends these instructions back to the server.

[0599] The server regenerates the 3D model based on the modification instructions, again using the generated AI model and prompts to make specific modifications to the model. The modified 3D model is then sent back to the user's device, and the process is repeated until the user is satisfied.

[0600] Final order confirmation

[0601] When the user is satisfied with the final 3D model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[0602] Project management and progress notifications

[0603] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check progress status such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color" through the app.

[0604] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[0605] Specific examples

[0606] Example 1:

[0607] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[0608] Example 2:

[0609] Another user requests a minimalist two-bedroom apartment, specifying their budget and materials to be used in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests changes to the bedroom location. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[0610] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[0611] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0612] Step 1:

[0613] The user downloads and installs the ordering app. The user launches the app and enters details such as name, contact information, desired home style, layout, budget, and materials on the basic information input screen.

[0614] Input: User information (name, contact details, house style, layout, budget, materials used, etc.)

[0615] Output: User information entered

[0616] Specific actions: The user operates a smartphone or tablet, fills in the required information in the information input form, and presses the submit button.

[0617] Step 2:

[0618] The terminal temporarily stores the entered user information in a local database and then transmits the information to the server.

[0619] Input: User information (obtained from local database)

[0620] Output: Data sent to the server (user information)

[0621] Specific operation: The device sends user information to the server using an HTTP request.

[0622] Step 3:

[0623] The server analyzes the received user information and generates an initial 3D model using generative AI (e.g., OpenAI's DALL-E or Autodesk's Revit).

[0624] Input: User information (analyzed by the server)

[0625] Output: Initial 3D model data

[0626] Specific operation: The server parses the user information and asks the generation AI to generate a 3D model using a prompt, such as "Please generate a 3D model of a natural modern style 4LDK house."

[0627] Step 4:

[0628] The server sends the generated 3D model data to the terminal and displays it to the user.

[0629] Input: Initial 3D model data

[0630] Output: Data sent to the device (initial 3D model)

[0631] Specific operation: The server returns the generated 3D model data to the terminal, and the terminal displays the 3D model to the user.

[0632] Step 5:

[0633] The user checks the 3D model displayed on the device and inputs any necessary corrections, such as "make the living space larger" or "change the location of the kitchen."

[0634] Input: Initial 3D model, user instructions for modification

[0635] Output: Correction instruction data

[0636] Specific action: The user enters the specific changes in text format into the correction instruction input form of the app and presses the submit button.

[0637] Step 6:

[0638] The terminal transmits a correction instruction to the server.

[0639] Input: Correction instruction data (obtained from the terminal)

[0640] Output: Data to be sent to the server (correction instructions)

[0641] Specific operation: The device sends correction instructions to the server using an HTTP request.

[0642] Step 7:

[0643] The server regenerates the 3D model based on the received modification instructions, and again uses a generation AI (such as OpenAI's DALL-E or Autodesk's Revit) to modify the model using prompts.

[0644] Input: Correction instruction data

[0645] Output: Regenerated 3D model data

[0646] Specific operation: The server analyzes the modification instructions and sends a regeneration prompt to the generation AI, such as "Please regenerate a 3D model of a 4LDK natural modern style house with an expanded living space."

[0647] Step 8:

[0648] The server transmits the regenerated 3D model data to the terminal and displays it to the user.

[0649] Input: Regenerated 3D model data

[0650] Output: Data sent to the device (corrected 3D model)

[0651] Specific operation: The server sends the regenerated 3D model to the device, which then displays it to the user. This process is repeated until the user is satisfied.

[0652] Step 9:

[0653] If the user is satisfied with the final model, he or she presses the order confirmation button.

[0654] Input: Final 3D model, order confirmation instructions

[0655] Output: Confirmed order data

[0656] Specific operation: The user presses the order confirmation button, and the terminal sends the confirmation data to the server.

[0657] Step 10:

[0658] The terminal transmits the final 3D model data and order information to the server.

[0659] Input: Confirmed order data

[0660] Output: Data sent to the server (final 3D model data, order information)

[0661] What happens: The device sends the final 3D model and order data to the server using an HTTP request.

[0662] Step 11:

[0663] The server transmits the received order data to affiliated construction companies and manages the project.

[0664] Input: Order data

[0665] Output: Data to be sent to construction companies (order data)

[0666] Specific operation: The server sends the order data to the construction company and registers it in the project management system.

[0667] Step 12:

[0668] The server notifies the user's terminal of the progress information of the project received from the construction company.

[0669] Input: Progress information (received from construction company)

[0670] Output: Notification data (progress information) sent to the user's device

[0671] Specific operation: The server periodically checks progress information and sends notifications to the user's device at appropriate times.

[0672] This allows users to move through the home design process efficiently and quickly, facilitating overall communication and management right up to the final order.

[0673] (Application example 1)

[0674] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] There is a demand for efficient design of the homes users desire and a highly reproducible virtual viewing experience. However, conventional systems were unable to quickly generate and modify 3D models that matched the user's wishes, and lacked a means to check the progress in real time. In addition, users could not actually view the home before it was completed, which could lead to low satisfaction with the final home design.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0677] In this invention, the server includes: means for a user to input the characteristics of a desired home; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model based on the analysis data using a generation AI; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; and means for displaying the 3D model in a VR space on the user terminal, allowing the user to virtually tour the home. This allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[0678] A "user terminal" is a device used by a user to input the characteristics of a house, check the 3D model, input correction instructions, and take a virtual tour.

[0679] The "server" is a device or system that receives and analyzes data sent from user devices and generates 3D models using generative AI. It also manages project progress information and notifies user devices.

[0680] "Generative AI" is an artificial intelligence model that automatically generates a 3D model based on the characteristic data of a house entered by the user.

[0681] A "3D model" is a digital model that represents the shape, layout, style, etc. of the home desired by the user in three dimensions.

[0682] "VR space" is a virtual space that uses virtual reality technology to allow users to tour homes in a virtual environment.

[0683] "Modification instructions" are input information that a user can use to request changes or improvements to specific parts of a 3D model after checking it.

[0684] A "virtual tour" is a virtual home tour method that allows users to use a VR space to actually walk around the generated 3D model and experience it for themselves.

[0685] "Progress information" is data that indicates the progress of construction and important events in a user's housing project.

[0686] MODE FOR CARRYING OUT THE INVENTION

[0687] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[0688] User requirements collection

[0689] First, users install a dedicated application on their smartphone. When they launch the application, a screen appears where they can enter details such as the desired style of the home (e.g., natural modern, classic, minimalist, etc.), floor plan, budget, and materials used. This data is temporarily stored on the user's device and then sent to the server.

[0690] 3D model generation and modification

[0691] The server receives and analyzes the data sent by the user. Based on the analysis results, the server uses a generative AI model to generate an initial 3D model. After generation, this 3D model is sent to the user's device and displayed in the application. The user reviews the 3D model and, if any modifications are necessary, enters specific instructions (e.g., "make the living room bigger"). Based on the modification instructions sent back to the server, the server regenerates a new 3D model and sends it back to the user's device. This process is repeated until the user is satisfied.

[0692] Virtual Tour

[0693] The user's device can display the generated 3D model in a VR space. The user can attach a smartphone to a VR headset and take a virtual tour. This allows the user to experience the interior of the designed home in real time and check the details.

[0694] Final order confirmation and construction management

[0695] Once the user is satisfied with the final model and places an order, the order data is sent to the server. The server aggregates the order information and sends it to the partner construction company. This process starts the construction project. The server periodically receives progress information from the construction company and notifies the user's device in real time. The user can check the project's progress (e.g., "foundation work completed") at any time through the application.

[0696] Specific examples

[0697] A user uses a smartphone app to input desired housing information. For example, they enter information such as "Style: Natural Modern," "Layout: 4LDK," "Budget: 50 million yen," and "Materials used: Wood, glass, concrete." The server receives this information and generates a 3D model using a generative AI model. The prompt statement "Please generate a 3D model of a 4LDK natural modern home based on the following information: Style: Natural Modern, Layout: 4LDK, Budget: 50 million yen, Materials used: Wood, glass, concrete" is passed to the generative AI model.

[0698] The user can view the generated 3D model in VR space, and if they feel there is a problem with the living space, they can input a correction instruction such as "make the living space larger." The server receives this correction instruction and updates the 3D model again using the generated AI model. This process is repeated until a model that satisfies the user is finally completed. Once the user confirms the final order, the construction project begins, and progress information is notified to the user in real time.

[0699] This system allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[0700] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0701] Step 1:

[0702] The user launches the smartphone app and inputs the desired housing characteristics (e.g., style, layout, budget, materials used, etc.). The input data is initially stored on the user's device and later sent to the server.

[0703] Input: House characteristic data entered by the user.

[0704] Output: House feature data temporarily stored on the user's device.

[0705] Step 2:

[0706] The device sends the user's input data to the server, which then analyzes the data. This analysis includes data validation and missing value completion.

[0707] Input: House characteristic data sent from the user terminal.

[0708] Output: Analyzed housing data stored on the server.

[0709] Step 3:

[0710] The server generates an initial 3D model using a generative AI model based on the analyzed data. A prompt is generated and input into the generative AI to generate the 3D model.

[0711] Input: Parsed housing data, prompt for the generative AI model.

[0712] Output: The generated 3D model data.

[0713] Step 4:

[0714] The server sends the generated 3D model data to the user's device, where the 3D model is displayed and the user can confirm the model.

[0715] Input: Generated 3D model data.

[0716] Output: The 3D model displayed on the user's device.

[0717] Step 5:

[0718] The user checks the 3D model and inputs correction instructions as necessary. The correction instructions are entered into the terminal as specific requests (e.g., "Make the living room bigger") and sent to the server.

[0719] Input: Correction instructions entered by the user.

[0720] Output: Correction instruction data sent to the server.

[0721] Step 6:

[0722] The server regenerates the 3D model based on the received correction instructions, and then passes the prompt sentence to the generative AI model again to generate an updated 3D model.

[0723] Input: User correction instruction data, update prompt sentence to the generative AI model.

[0724] Output: Updated 3D model data.

[0725] Step 7:

[0726] The server then sends the updated 3D model back to the user's device, where the user reviews the new 3D model and repeats this process until the modifications are complete.

[0727] Input: Updated 3D model data.

[0728] Output: The updated 3D model displayed on the user's device.

[0729] Step 8:

[0730] When the user finally completes a model that satisfies them, they press the order confirmation button, and the terminal sends the confirmed order data to the server.

[0731] Input: Final confirmed 3D model, order confirmation information.

[0732] Output: Order confirmation data sent to the server.

[0733] Step 9:

[0734] The server sends the order information to the construction company based on the received order confirmation data. It also starts managing the construction project, periodically receives progress information from the construction company, and notifies the user terminal.

[0735] Input: Order confirmation data, project progress information.

[0736] Output: Order information sent to the construction company, progress information notified to the user terminal.

[0737] Step 10:

[0738] The 3D model is displayed in a VR space so that users can take a virtual tour through their devices. Users can connect their smartphone to a VR headset and enjoy the virtual tour experience.

[0739] Input: 3D model data to be displayed in VR space.

[0740] Output: 3D model displayed in a VR headset for virtual tours.

[0741] Through these steps, users can design their desired home in detail, check, modify, and finalize the design in virtual space, and keep track of the project's progress in real time.

[0742] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0743] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[0744] User requirement collection and emotion recognition

[0745] A user downloads and installs the ordering app. When the user launches the app, a screen for entering basic information appears. The ordering app has the ability to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions during the process.

[0746] Generating 3D models and reflecting emotion data

[0747] The device sends the user's input data and the recognized emotion data to the server. The server then first analyzes the characteristic data of the user's desired home, and then analyzes the emotion data. Based on the analysis results, the generative AI then activates and generates an initial 3D model that takes the user's emotion data into account. This process makes it possible, for example, to design a home that incorporates elements that the user feels positive about.

[0748] Emotional reflection of 3D model checking and correction

[0749] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs corrections they would like to make to the model, their emotions are also collected at the same time. For example, along with instructions such as "I would like the living space to be a little larger," the user's emotions at that time (dissatisfaction, expectations, etc.) are sent.

[0750] The device sends these correction instructions and emotion data to the server, which then uses the generative AI to correct the 3D model. The emotion data is used to predict the importance of the user's request and makes optimal corrections based on that.

[0751] Final order confirmation

[0752] When the user is satisfied with the final model, they press the order confirmation button. The device then sends the final 3D model data, order information, and emotion data to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is then sent to the device as a notification to the user.

[0753] Emotional reflection in project management and progress notifications

[0754] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at the set timing. The emotion engine also operates when notifications are sent, capturing the user's emotions when providing feedback and sending them to the server.

[0755] Specific examples

[0756] Example 1:

[0757] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[0758] Example 2:

[0759] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[0760] In this way, by incorporating an emotion engine, the system of the present invention can design a house that reflects the user's feelings, thereby achieving higher customer satisfaction.

[0761] The processing flow will be explained below.

[0762] Step 1:

[0763] A user downloads and installs the ordering app. When the user launches the app, a screen appears where the user can enter their information (name, contact details).

[0764] Step 2:

[0765] In addition to a screen for entering basic information, the device also activates an emotion engine that captures the user's facial expressions and voice. As the user enters information, the emotion engine captures the user's emotional data in real time.

[0766] Step 3:

[0767] The user inputs the home's characteristics (style, layout, budget, materials used, etc.) into the ordering app. The emotion engine captures the user's emotions while they are typing.

[0768] Step 4:

[0769] The device temporarily stores the user's input data and emotion data in a local database and then transmits it to the server.

[0770] Step 5:

[0771] The server receives and analyzes the user's input data, then analyzes the emotion data. Based on these analysis results, a generative AI is activated to generate an initial 3D model that takes the user's emotions into account.

[0772] Step 6:

[0773] The server sends the generated 3D model and the emotion analysis results to the device, which then displays the 3D model to the user and asks for their confirmation.

[0774] Step 7:

[0775] The user checks the 3D model and inputs the parts they want to modify. At the same time, the emotion engine captures the user's emotions and sends them, along with the modification instructions, from the device to the server.

[0776] Step 8:

[0777] The server receives the modification instructions and emotion data, and again modifies the 3D model using the generative AI. At this time, the model is adjusted with an emphasis on the user's emotion data. For example, it focuses on improving areas that the user expressed dissatisfaction with.

[0778] Step 9:

[0779] The server resends the modified 3D model to the device, which displays it to the user and asks for confirmation again. This process is repeated until the user is satisfied.

[0780] Step 10:

[0781] If the user is satisfied with the final model, they press the "Confirm Order" button, and the device sends the final 3D model data, order information, and emotion data to the server.

[0782] Step 11:

[0783] The server collects the order details and sends the order data to the affiliated construction company. This confirmation information is then sent to the terminal and notified to the user.

[0784] Step 12:

[0785] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[0786] Step 13:

[0787] The server periodically sends progress information to the device. The device notifies the user of the progress and displays it. The emotion engine also works when notifying the user, capturing the emotions expressed by the user regarding the progress.

[0788] Step 14:

[0789] Users can check the progress and enter feedback as needed. For example, requests such as "I want to change the color of the exterior walls" are captured along with their emotions at the time.

[0790] Step 15:

[0791] The device sends the user's feedback and emotional data to a server, which analyzes the feedback and forwards it to the construction company.

[0792] Step 16:

[0793] The construction company makes corrections based on the feedback and performs a final check. The server sends a completion notification to the terminal and notifies the user of the date and time of handover. The user makes a final check and the handover is completed in a satisfactory manner.

[0794] Example 2

[0795] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0796] Conventional home design systems can collect specific user requirements, but do not consider the emotions felt by the user during the design process. As a result, the user's emotions and satisfaction are not reflected in the design process, which can lead to dissatisfaction with the final home design. In particular, there is a need for a system that can accurately reflect the user's emotions when making changes to the layout or style of the home.

[0797] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0798] In this invention, the server includes: a means for a user to input the characteristics of a home desired by the user; a means for the terminal to capture the user's facial expressions and voice in real time and recognize emotions; a means for the server to receive and analyze the user's input data and emotional data; a means for the server to generate a 3D model using a generation AI based on the analyzed data; a means for the server to send the generated 3D model data to the terminal; a means for the user to check the 3D model and input correction instructions; a means for the terminal to recapture the user's facial expressions and voice and recognize emotions; a means for the server to receive the correction instructions and emotional data and regenerate the 3D model; a means for the user to make a final confirmation and confirm the order; a means for the server to send the order data to a construction company and manage the project; and a means for the server to notify the terminal of project progress information. This makes it possible to reflect the user's emotions in the home design process and provide a final home design that will provide a high level of satisfaction.

[0799] "User" refers to an individual or corporation that uses the system to design a desired home.

[0800] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[0801] "Facial and audio capture" refers to the process by which the device records the user's facial movements and voice in real time.

[0802] "Emotion recognition" refers to analyzing a user's emotional state (e.g., joy, sadness, anticipation, dissatisfaction) from captured facial and voice data.

[0803] "Server" refers to a central computing device that processes and manages data for the entire system.

[0804] "Input data" refers to information including the characteristics and requirements of the home desired by the user.

[0805] "Emotion data" refers to information about emotions recognized from the user's facial expressions and voice.

[0806] "Analyzing" refers to the server processing the received data with an analysis algorithm.

[0807] "Generative AI" refers to a system that uses artificial intelligence technology to generate a 3D model of a house based on user requirements and emotional data.

[0808] "3D model" refers to three-dimensional design data for a house that is generated based on the user's wishes and emotional data.

[0809] "Sending" refers to the process by which the server transfers the generated data to the terminal.

[0810] "Modification instructions" refer to requests for changes or adjustments made by the user to the 3D model.

[0811] "Regenerate" refers to the server recreating the 3D model based on the correction instructions and emotion data.

[0812] "Final confirmation" refers to when the user is satisfied with the final 3D model and confirms the order.

[0813] "Order Data" means all order information included with the final confirmed 3D model.

[0814] "Construction company" refers to the corporation that actually constructs the housing ordered.

[0815] "Managing the project" refers to the server monitoring and managing the progress of the entire home building process.

[0816] "Progress Information" means data relating to the progress of a construction project.

[0817] "Notifying" refers to the process by which the server communicates progress and other important information to the device.

[0818] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[0819] User requirement collection and emotion recognition

[0820] A user downloads and installs a home design app on their smartphone or tablet. When the app is launched, a screen for entering basic information appears. The app is equipped with a function to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions in real time. Specifically, the app uses the smartphone's camera and microphone to acquire emotion data using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[0821] Generating 3D models and reflecting emotion data

[0822] The device sends the user's input data and emotion data to the server. Upon receiving this data, the server first analyzes the user's request data, and then analyzes the emotion data. Based on the analysis results, the server activates the generative AI to generate an initial 3D model that reflects the user's emotion data. The server then sends the generated 3D model data to the device, and uses an interactive viewer (e.g., Three.js) to display the 3D model on the smartphone screen.

[0823] Emotional reflection of 3D model checking and correction

[0824] When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs the parts of the model they want to modify, their emotions are also collected at the same time. Specifically, the camera and microphone are used again to collect the user's emotional data. The device sends these modification instructions and emotional data to the server, and the server again uses the generative AI to modify the 3D model. The emotional data is used to determine the importance of the user's request, and the optimal modifications are made based on that.

[0825] Final order confirmation and project management

[0826] When the user is satisfied with the final model, they press the order confirmation button. The device sends the final 3D model data, order information, and emotion data to the server. The server consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user. Once the project begins, the server manages the progress of the construction project. The construction company periodically reports progress information to the server, and the server receives this progress information and notifies the device at the set time. The emotion engine also runs when notifying, capturing the emotion of the user's feedback and sending it to the server.

[0827] Examples and thought-provoking prompts

[0828] Example 1

[0829] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[0830] Prompt Sentence Examples

[0831] If a user looking for a natural modern 4LDK home expresses dissatisfaction with the size of the living space, please suggest how to improve it based on emotional data.

[0832] Example 2

[0833] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[0834] Prompt Sentence Examples

[0835] If a user with a budget constraint requests to change the location of the bedroom in a minimalist 2LDK apartment, please suggest the optimal layout based on the sentiment data.

[0836] In this way, by incorporating an emotion engine, the system of the present invention can design a home that reflects the user's feelings, achieving higher customer satisfaction. The hardware used to implement this system includes a smartphone and a server, and specific software includes facial recognition software (e.g., OpenFace), voice analysis software (e.g., IBM Watson), and an interactive viewer (e.g., Three.js).

[0837] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0838] Step 1:

[0839] A user downloads and installs a home design app.

[0840] What it does: Download and install the app from the app store on your smartphone or tablet.

[0841] Input: None

[0842] Output: Device with home design app installed

[0843] Step 2:

[0844] The user launches the app and is presented with a screen for entering basic information.

[0845] Specific operation: When the user taps the app icon to launch the app, a screen for entering basic information appears on the device.

[0846] Input: None

[0847] Output: Basic information input screen

[0848] Step 3:

[0849] Users enter details such as their name, contact information, desired home style and layout, budget, and materials used.

[0850] Specific behavior: The user enters the required information into the form and proceeds to the next screen.

[0851] Input: User-entered house characteristics data

[0852] Output: The entered house feature data is saved on the device.

[0853] Step 4:

[0854] The device captures the user's facial expressions and voice in real time and recognizes their emotions.

[0855] Specific operation: The smartphone's camera and microphone are activated, and emotion data is obtained using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[0856] Input: User's facial expression and voice data

[0857] Output: Emotion data

[0858] Step 5:

[0859] The terminal transmits the user's input data and emotion data to the server.

[0860] Specific behavior: The input data and captured emotion data are sent to the server using the HTTPS protocol.

[0861] Input: House feature data, emotion data

[0862] Output: Data sent to the server

[0863] Step 6:

[0864] The server analyzes the received data and generates an initial 3D model using a generative AI model.

[0865] Specific operation: The server analyzes the received data and obtains the analysis results using a deep learning model (e.g., TensorFlow). Based on that data, it uses generative AI (e.g., Unity3D or Blender) to render a 3D model.

[0866] Input: House feature data, emotion data

[0867] Output: Initial 3D model

[0868] Step 7:

[0869] The server sends the generated 3D model data to the terminal.

[0870] Specific operation: The generated 3D model data is encoded and sent to the terminal using the HTTPS protocol.

[0871] Input: Initial 3D model

[0872] Output: 3D model sent to device

[0873] Step 8:

[0874] The device displays the 3D model to the user.

[0875] Specific operation: Use an interactive viewer (e.g., Three.js) to display a 3D model on the smartphone screen and allow the user to check it.

[0876] Input: 3D model data

[0877] Output: A displayed 3D model

[0878] Step 9:

[0879] The user checks the 3D model and inputs correction instructions.

[0880] Specific operation: The user adjusts the model using the touch interface of the smartphone.

[0881] Input: User correction instructions

[0882] Output: Correction instruction data

[0883] Step 10:

[0884] The device recaptures the user's facial expressions and voice to recognize emotions.

[0885] Specific operation: Again, the camera and microphone are used to collect user emotion data.

[0886] Input: User's facial expression and voice data

[0887] Output: Emotion data at the time of correction

[0888] Step 11:

[0889] The device sends correction instructions and emotion data to the server.

[0890] Specific operation: Input data and emotion data are sent to the server using the HTTPS protocol.

[0891] Input: Correction instruction data, emotion data when correcting

[0892] Output: Correction data and emotion data sent to the server

[0893] Step 12:

[0894] The server regenerates the 3D model based on the correction instructions and emotion data.

[0895] Specific operation: The server integrates the correction instructions and emotion data, and again uses the generative AI to generate a corrected 3D model.

[0896] Input: Correction instruction data, emotion data when correcting

[0897] Output: Modified 3D model

[0898] Step 13:

[0899] If the user is satisfied with the final model, he or she presses the order confirmation button.

[0900] Specific operation: The user presses the order confirmation button on the smartphone.

[0901] Input: User order confirmation operation

[0902] Output: Order confirmation data

[0903] Step 14:

[0904] The terminal sends the final 3D model data, order information, and emotion data to the server.

[0905] Specific behavior: The final data is sent to the server using the HTTPS protocol.

[0906] Input: Final 3D model data, order information, emotion data

[0907] Output: Final data sent to the server

[0908] Step 15:

[0909] The server collects the order information and sends the order data to the affiliated construction company.

[0910] Specific operation: The server sends the order data to the construction company via email or API.

[0911] Input: Order details data

[0912] Output: Order data sent to construction company

[0913] Step 16:

[0914] The server notifies the user of the order confirmation information.

[0915] Specific operation: The server sends a push notification to the device to notify the user that the order has been confirmed.

[0916] Input: Order confirmation information

[0917] Output: Push notification to the user

[0918] Step 17:

[0919] The server manages the progress of the construction project.

[0920] What it does: The server uses project management software (e.g. Jira) to track progress.

[0921] Input: Project Management Information

[0922] Output: Managed progress data

[0923] Step 18:

[0924] The construction company reports progress information to the server.

[0925] What happens: A construction company uses a progress reporting tool (e.g., Microsoft Project) to send progress data to a server.

[0926] Input: Progress information

[0927] Output: Progress data sent to the server

[0928] Step 19:

[0929] The server notifies the user of progress information.

[0930] Specific operation: The server sends a push notification to the device at the set timing.

[0931] Input: Progress information

[0932] Output: Progress notification to user

[0933] Step 20:

[0934] The device will display a notification and the emotion engine will capture the emotion during the feedback.

[0935] Specific behavior: When the user confirms the notification, the device again uses the camera and microphone to capture emotion data.

[0936] Input: Emotional data as user feedback

[0937] Output: Feedback emotion data sent to the server

[0938] (Application example 2)

[0939] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0940] The current home design process rarely takes into account user emotions, making it difficult to increase true user satisfaction. Virtual stores also lack personalized responses based on user emotions, creating a need for improved user experience. Conventional technologies lack the means to analyze user emotions in real time and customize products and services based on those emotions, making user satisfaction a challenge.

[0941] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input the characteristics of the house desired by the user; means for the terminal to capture the user's facial expressions and voice and acquire emotional data; means for the server to receive and analyze the user's input data and emotional data; means for the server to generate a 3D model based on the analyzed data and taking the emotional data into consideration using a generation AI; means for the server to send the generated 3D model data to the user terminal; means for the user to check the 3D model and input correction instructions; means for the terminal to re-capture the user's facial expressions and voice and acquire emotional data; means for the server to receive the correction instructions and emotional data and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to send the order data to the construction company and manage the project; and means for the server to notify the user terminal of project progress information. This makes it possible to provide more personalized products and services that reflect the user's emotions in real time.

[0942] The "means for the user to input the characteristics of the home desired" is a device or system that provides an interface for the user to specifically input the requirements and desires regarding the ideal home.

[0943] "Means for the device to capture the user's facial expressions and voice and obtain emotional data" refers to technology that captures the user's facial expressions and voice in real time, analyzes them, and obtains the user's emotions as data.

[0944] The "means for the server to receive and analyze the input data and emotion data of the user" refers to a device or system that receives data related to housing requirements and emotions sent from the user and performs an analysis process.

[0945] "Means for the server to use generative AI based on the analyzed data to generate a 3D model that also takes into account emotional data" refers to a technology that uses generative AI to create a 3D model that reflects the user's emotions based on analyzed data.

[0946] "Means by which the server transmits the generated 3D model data to the user terminal" refers to a mechanism for transferring the generated 3D model data to the terminal operated by the user.

[0947] "Means for the user to check the 3D model and input correction instructions" refers to a device or system that displays the 3D model to the user and provides an interface that allows the user to check it and then input correction instructions.

[0948] "Means for the device to recapture the user's facial expressions and voice and obtain emotional data" refers to technology that recaptures the user's facial expressions and voice and digitizes their emotions when checking and correcting the 3D model.

[0949] "Means for the server to receive correction instructions and emotion data and regenerate the 3D model" refers to technology that regenerates the 3D model based on correction instructions sent by the user and re-acquired emotion data.

[0950] "Means for the user to make final confirmation and confirm the order" refers to the procedures and devices by which the user can finally confirm the 3D model and officially confirm the order after being satisfied with it.

[0951] The "means for the server to transmit order data to the construction company and manage the project" is a system for transmitting confirmed order data to the construction company and managing the progress of the project.

[0952] "Means for the server to notify the user terminal of project progress information" refers to a technology that periodically notifies the terminal operated by the user of the progress status of the project.

[0953] The present invention is a system that recognizes a user's emotions and reflects them in the home design process, enabling more satisfying home designs. It can also be applied as a system for suggesting products in a virtual store based on the user's emotions. Detailed embodiments of the system are described below.

[0954] System Program

[0955] This system recognizes the user's emotions and analyzes their emotional data to suggest the most suitable 3D models and products for the user. This process includes the following elements:

[0956] 1. Obtaining user input and emotion data:

[0957] Users download a dedicated application and input the desired characteristics for their home design. The camera and microphone on their device (such as a smartphone or tablet) capture the user's facial expressions and voice in real time to obtain emotional data. The DeepFace library is used to analyze facial expressions and recognize key emotions (e.g., "happiness," "sadness," "surprise," etc.).

[0958] 2. Data Receipt and Analysis:

[0959] The input data and emotion data acquired by the device are sent to a server, where they are simultaneously analyzed and home design data that reflects the user's emotions is generated. This analysis is performed using machine learning models and generative AI.

[0960] 3. Generate 3D model:

[0961] Using generative AI, an initial 3D model is generated that takes into account the user's emotional data. This generation takes into account the user's desired home characteristics data and the results of emotional analysis.

[0962] 4. Check and correct the 3D model:

[0963] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model and inputs correction instructions or desired changes, the device again captures facial expressions and voice to obtain emotional data. This allows the device to analyze the importance of the correction instructions and the intensity of the emotion, allowing for more appropriate corrections.

[0964] 5. Final model confirmation and project management:

[0965] Once the user is satisfied with the final 3D model, they confirm their order. The server sends the order data to the construction company and manages the project progress. Project progress information is periodically sent to the user's device, and emotional data is also collected during the project.

[0966] Hardware and software used

[0967] Hardware:

[0968] Smartphone, tablet, or PC camera

[0969] microphone

[0970] software:

[0971] OpenCV

[0972] DeepFace

[0973] Hugging Face transformers library

[0974] Specific examples

[0975] 1. User's house design:

[0976] A user looking for a natural modern-style home inputs their information into the application. As they input, the camera and microphone capture their facial expressions and voice, and the emotion engine recognizes that the user is expressing strong expectations for the living space. Based on this, the generative AI creates an initial 3D model and presents it to the user.

[0977] Example prompt sentence:

[0978] "Users are expressing joy. Please suggest appropriate living space layouts."

[0979] 2. Product suggestions in virtual stores:

[0980] The facial expressions and voices of users shopping using a virtual store application are captured, and if the user expresses feelings of fatigue, the generative AI will suggest relaxing products (e.g., aroma candles).

[0981] Example prompt sentence:

[0982] "The user is showing signs of fatigue. Please suggest appropriate relaxation items."

[0983] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0984] Step 1:

[0985] A user downloads and installs a home design app. When the app is launched, a screen for entering basic information appears. While the user enters the desired home characteristics (style, layout, budget, materials used, etc.), the device's (smartphone or tablet) camera and microphone capture the user's facial expressions and voice in real time, obtaining emotion data. This data is obtained using the OpenCV and DeepFace libraries. The input is the user's information, facial expressions, and voice data, and the output is emotion data.

[0986] Step 2:

[0987] The input data (house features) and emotion data acquired by the device are sent to the server. The server receives the sent data and performs an analysis process. The server analyzes the house data entered by the user and the captured emotion data to identify the house features desired by the user. The input is the house features and emotion data, and the output is the analyzed feature data.

[0988] Step 3:

[0989] The server uses a generative AI based on the analyzed data to generate an initial 3D model that also takes into account emotional data. The Hugging Face generative AI model is used for this generation. The analysis results and a prompt are input into the generative AI model, which outputs a 3D model that reflects the user's emotions. The input is the analyzed feature data, emotional data, and the prompt "Please propose the layout of the living space that the user desires," and the output is the initial 3D model.

[0990] Step 4:

[0991] The server sends the generated 3D model data to the user's device. The user checks the 3D model on the device and inputs any complaints or corrections. At this time, the device again captures the user's facial expressions and voice, and obtains emotional data accompanying the corrections. The input is the 3D model data, and the output is the corrections and emotional data.

[0992] Step 5:

[0993] The device sends the correction instructions and emotion data to the server. The server receives them and regenerates the 3D model using the analysis and generative AI. The input is the correction instructions, emotion data, and the prompt "Please suggest the optimal layout of the living space based on the user's correction instructions." The output is the corrected 3D model.

[0994] Step 6:

[0995] Once the user has finalized the 3D model that satisfies them, they confirm their order. The terminal sends the final model data and order information to the server. The input is the confirmed 3D model and order information, and the output is the final order data.

[0996] Step 7:

[0997] The server sends order data to the construction company and manages the progress of the project. Progress information on ongoing projects is periodically reported to the server, which then notifies the user's device. When notifying the user, the device also captures the user's facial expression and collects emotional data. The input is progress information from the construction company, and the output is progress notification and emotional data to the user.

[0998] This series of processes enables the design of homes that reflect the user's emotions and provide a higher level of satisfaction. Furthermore, emotion-based product suggestions can be made as needed.

[0999] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1000] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1001] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1002] [Third embodiment]

[1003] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1004] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1005] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1006] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1007] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1008] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1009] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1010] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1011] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1012] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1013] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1014] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1015] The present invention is a system for efficiently designing and quickly providing a home that a user desires. This system inputs the characteristics of the home that the user desires, analyzes that information, and generates a 3D model of the home. Details of the system and its embodiments are described below.

[1016] User requirements collection

[1017] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen, where they enter details such as their name, contact information, desired home style (e.g., natural modern, minimalist, classic, etc.), floor plan, budget, and materials used.

[1018] 3D model generation and modification

[1019] The device temporarily stores the information entered by the user in a local database and then sends it to the server. The server then analyzes the data and uses generative AI to generate an initial 3D model based on the user's requests.

[1020] The generated 3D model is sent to the device and displayed to the user. The user reviews the 3D model and indicates any necessary modifications. For example, they can input specific requests such as "I want to make the living space larger" or "I want to change the location of the kitchen." The device then sends the modifications back to the server, which then regenerates the 3D model based on those instructions. This modification process is repeated until the user is satisfied.

[1021] Final order confirmation

[1022] When the user is satisfied with the final model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[1023] Project management and progress notifications

[1024] Once the order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check the progress status through the app, such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color."

[1025] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[1026] Specific examples

[1027] Example 1:

[1028] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[1029] Example 2:

[1030] Another user requests a minimalist 2LDK layout. They set their budget and materials in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests a change in the location of the bedroom. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[1031] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[1032] The processing flow will be explained below.

[1033] Step 1:

[1034] The user downloads and installs the ordering app. The user launches the app and enters their user information (name, contact details).

[1035] Step 2:

[1036] The terminal displays a form for the user to input the characteristics of the home they desire (style, layout, budget, materials, etc.). The user enters this information.

[1037] Step 3:

[1038] The terminal temporarily stores the user's input data in a local database and then transmits it to the server.

[1039] Step 4:

[1040] The server receives and analyzes the user's input data. Based on the analysis results, it activates the generation AI to generate the initial 3D model.

[1041] Step 5:

[1042] The server sends the generated 3D model to the device, which displays the 3D model to the user and asks for confirmation.

[1043] Step 6:

[1044] The user checks the 3D model and inputs any modifications they would like to make, such as making the living space larger.

[1045] Step 7:

[1046] The device sends the user's correction instructions back to the server, which receives the instructions, uses the generation AI to correct the 3D model again, and sends it back to the device.

[1047] Step 8:

[1048] This process is repeated several times until a final model that satisfies the user is determined, and the user presses the "Confirm Order" button.

[1049] Step 9:

[1050] The terminal sends the final 3D model data and order information to the server, which then aggregates the order details and sends the order data to the construction company.

[1051] Step 10:

[1052] The server sends a notification of order confirmation to the terminal, which receives the notification and displays it to the user.

[1053] Step 11:

[1054] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[1055] Step 12:

[1056] The server periodically sends progress information to the terminal, which then notifies and displays the progress information to the user.

[1057] Step 13:

[1058] The user can check the progress and enter feedback as needed, for example, a request such as "I would like to change the color of the exterior walls."

[1059] Step 14:

[1060] The device sends the feedback to the server, which forwards it to the construction company.

[1061] Step 15:

[1062] The construction company makes corrections based on the feedback and performs a final check. The server notifies the user of completion and the date and time of handover.

[1063] Step 16:

[1064] The user makes a final check and the delivery is completed in a satisfactory manner.

[1065] Example 1

[1066] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] The traditional residential design process required a great deal of time and effort to meet the detailed needs of the user, making it difficult to design efficiently. Furthermore, there was a lack of collaboration and communication between the user and the construction company, making it difficult to make design changes or check progress. This could lead to lower customer satisfaction and project delays.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1069] In this invention, the server includes: means for a user to input the characteristics of a home desired by the user; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model using a generative AI based on the analyzed data; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; means for the user to input order and correction data through an application; means for the server to operate the generative AI model using prompt sentences based on the user's input data; and means for the terminal to transmit order confirmation information from the user to the server. This enables home design that can quickly and efficiently meet the user's detailed needs, facilitating project progress management and communication with users.

[1070] A "user" is an individual or corporation that wishes to design and order a home.

[1071] An "ordering app" is a software application that allows users to enter the information necessary to design and order a home.

[1072] The "server" is a computer system that has the function of analyzing data collected from users, generating and modifying 3D models using generative AI, and finally transmitting order data to construction companies.

[1073] "Terminal" refers to a device that allows a user to input information via the ordering app, check the 3D model, and input correction instructions.

[1074] A "construction company" is a company or organization that actually constructs a house based on the order data sent from the server.

[1075] "Generative AI" is an artificial intelligence technology that automatically generates and modifies 3D models of homes based on user input data.

[1076] "3D Model" means a three-dimensional digital model of a house generated based on a user's requirements.

[1077] "Modification instructions" are input information for changes or adjustments that the user makes to the 3D model after checking it.

[1078] A "prompt" is a textual instruction used by the server to operate a generative AI model.

[1079] "Project progress information" is data that indicates the status and progress of housing projects currently underway by construction companies.

[1080] "Feedback" refers to additional requests or change requests made by users during the progress of a construction project.

[1081] "Confirmed order information" is the final confirmation data that is sent when the user is satisfied with the final model and officially orders the house.

[1082] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[1083] User requirements collection

[1084] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen where they enter details such as their name, contact information, desired home style (natural modern, minimalist, classic, etc.), floor plan, budget, and materials used. Information is entered using a device such as a smartphone or tablet.

[1085] 3D model generation and modification

[1086] The device temporarily stores the information entered by the user in a local database and then sends it to a server. The server used here is, for example, a cloud-based server system. The server that receives the data first analyzes the data and then uses a generative AI (such as OpenAI's DALL-E or Autodesk's Revit) to generate an initial 3D model. This 3D model is generated based on the user's requirements.

[1087] The generated 3D model is sent to the device and displayed to the user. The user checks the 3D model and, if any modifications are needed, inputs specific instructions. For example, instructions might be to "make the living space larger" or "change the location of the kitchen." The device then sends these instructions back to the server.

[1088] The server regenerates the 3D model based on the modification instructions, again using the generated AI model and prompts to make specific modifications to the model. The modified 3D model is then sent back to the user's device, and the process is repeated until the user is satisfied.

[1089] Final order confirmation

[1090] When the user is satisfied with the final 3D model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[1091] Project management and progress notifications

[1092] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check progress status such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color" through the app.

[1093] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[1094] Specific examples

[1095] Example 1:

[1096] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[1097] Example 2:

[1098] Another user requests a minimalist two-bedroom apartment, specifying their budget and materials to be used in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests changes to the bedroom location. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[1099] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[1100] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1101] Step 1:

[1102] The user downloads and installs the ordering app. The user launches the app and enters details such as name, contact information, desired home style, layout, budget, and materials on the basic information input screen.

[1103] Input: User information (name, contact details, house style, layout, budget, materials used, etc.)

[1104] Output: User information entered

[1105] Specific actions: The user operates a smartphone or tablet, fills in the required information in the information input form, and presses the submit button.

[1106] Step 2:

[1107] The terminal temporarily stores the entered user information in a local database and then transmits the information to the server.

[1108] Input: User information (obtained from local database)

[1109] Output: Data sent to the server (user information)

[1110] Specific operation: The device sends user information to the server using an HTTP request.

[1111] Step 3:

[1112] The server analyzes the received user information and generates an initial 3D model using generative AI (e.g., OpenAI's DALL-E or Autodesk's Revit).

[1113] Input: User information (analyzed by the server)

[1114] Output: Initial 3D model data

[1115] Specific operation: The server parses the user information and asks the generation AI to generate a 3D model using a prompt, such as "Please generate a 3D model of a natural modern style 4LDK house."

[1116] Step 4:

[1117] The server sends the generated 3D model data to the terminal and displays it to the user.

[1118] Input: Initial 3D model data

[1119] Output: Data sent to the device (initial 3D model)

[1120] Specific operation: The server returns the generated 3D model data to the terminal, and the terminal displays the 3D model to the user.

[1121] Step 5:

[1122] The user checks the 3D model displayed on the device and inputs any necessary corrections, such as "make the living space larger" or "change the location of the kitchen."

[1123] Input: Initial 3D model, user instructions for modification

[1124] Output: Correction instruction data

[1125] Specific action: The user enters the specific changes in text format into the correction instruction input form of the app and presses the submit button.

[1126] Step 6:

[1127] The terminal transmits a correction instruction to the server.

[1128] Input: Correction instruction data (obtained from the terminal)

[1129] Output: Data to be sent to the server (correction instructions)

[1130] Specific operation: The device sends correction instructions to the server using an HTTP request.

[1131] Step 7:

[1132] The server regenerates the 3D model based on the received modification instructions, and again uses a generation AI (such as OpenAI's DALL-E or Autodesk's Revit) to modify the model using prompts.

[1133] Input: Correction instruction data

[1134] Output: Regenerated 3D model data

[1135] Specific operation: The server analyzes the modification instructions and sends a regeneration prompt to the generation AI, such as "Please regenerate a 3D model of a 4LDK natural modern style house with an expanded living space."

[1136] Step 8:

[1137] The server transmits the regenerated 3D model data to the terminal and displays it to the user.

[1138] Input: Regenerated 3D model data

[1139] Output: Data sent to the device (corrected 3D model)

[1140] Specific operation: The server sends the regenerated 3D model to the device, which then displays it to the user. This process is repeated until the user is satisfied.

[1141] Step 9:

[1142] If the user is satisfied with the final model, he or she presses the order confirmation button.

[1143] Input: Final 3D model, order confirmation instructions

[1144] Output: Confirmed order data

[1145] Specific operation: The user presses the order confirmation button, and the terminal sends the confirmation data to the server.

[1146] Step 10:

[1147] The terminal transmits the final 3D model data and order information to the server.

[1148] Input: Confirmed order data

[1149] Output: Data sent to the server (final 3D model data, order information)

[1150] What happens: The device sends the final 3D model and order data to the server using an HTTP request.

[1151] Step 11:

[1152] The server transmits the received order data to affiliated construction companies and manages the project.

[1153] Input: Order data

[1154] Output: Data to be sent to construction companies (order data)

[1155] Specific operation: The server sends the order data to the construction company and registers it in the project management system.

[1156] Step 12:

[1157] The server notifies the user's terminal of the progress information of the project received from the construction company.

[1158] Input: Progress information (received from construction company)

[1159] Output: Notification data (progress information) sent to the user's device

[1160] Specific operation: The server periodically checks progress information and sends notifications to the user's device at appropriate times.

[1161] This allows users to move through the home design process efficiently and quickly, facilitating overall communication and management right up to the final order.

[1162] (Application example 1)

[1163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1164] There is a demand for efficient design of the homes users desire and a highly reproducible virtual viewing experience. However, conventional systems were unable to quickly generate and modify 3D models that matched the user's wishes, and lacked a means to check the progress in real time. In addition, users could not actually view the home before it was completed, which could lead to low satisfaction with the final home design.

[1165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1166] In this invention, the server includes: means for a user to input the characteristics of a desired home; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model based on the analysis data using a generation AI; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; and means for displaying the 3D model in a VR space on the user terminal, allowing the user to virtually tour the home. This allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[1167] A "user terminal" is a device used by a user to input the characteristics of a house, check the 3D model, input correction instructions, and take a virtual tour.

[1168] The "server" is a device or system that receives and analyzes data sent from user devices and generates 3D models using generative AI. It also manages project progress information and notifies user devices.

[1169] "Generative AI" is an artificial intelligence model that automatically generates a 3D model based on the characteristic data of a house entered by the user.

[1170] A "3D model" is a digital model that represents the shape, layout, style, etc. of the home desired by the user in three dimensions.

[1171] "VR space" is a virtual space that uses virtual reality technology to allow users to tour homes in a virtual environment.

[1172] "Modification instructions" are input information that a user can use to request changes or improvements to specific parts of a 3D model after checking it.

[1173] A "virtual tour" is a virtual home tour method that allows users to use a VR space to actually walk around the generated 3D model and experience it for themselves.

[1174] "Progress information" is data that indicates the progress of construction and important events in a user's housing project.

[1175] MODE FOR CARRYING OUT THE INVENTION

[1176] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[1177] User requirements collection

[1178] First, users install a dedicated application on their smartphone. When they launch the application, a screen appears where they can enter details such as the desired style of the home (e.g., natural modern, classic, minimalist, etc.), floor plan, budget, and materials used. This data is temporarily stored on the user's device and then sent to the server.

[1179] 3D model generation and modification

[1180] The server receives and analyzes the data sent by the user. Based on the analysis results, the server uses a generative AI model to generate an initial 3D model. After generation, this 3D model is sent to the user's device and displayed in the application. The user reviews the 3D model and, if any modifications are necessary, enters specific instructions (e.g., "make the living room bigger"). Based on the modification instructions sent back to the server, the server regenerates a new 3D model and sends it back to the user's device. This process is repeated until the user is satisfied.

[1181] Virtual Tour

[1182] The user's device can display the generated 3D model in a VR space. The user can attach a smartphone to a VR headset and take a virtual tour. This allows the user to experience the interior of the designed home in real time and check the details.

[1183] Final order confirmation and construction management

[1184] Once the user is satisfied with the final model and places an order, the order data is sent to the server. The server aggregates the order information and sends it to the partner construction company. This process starts the construction project. The server periodically receives progress information from the construction company and notifies the user's device in real time. The user can check the project's progress (e.g., "foundation work completed") at any time through the application.

[1185] Specific examples

[1186] A user uses a smartphone app to input desired housing information. For example, they enter information such as "Style: Natural Modern," "Layout: 4LDK," "Budget: 50 million yen," and "Materials used: Wood, glass, concrete." The server receives this information and generates a 3D model using a generative AI model. The prompt statement "Please generate a 3D model of a 4LDK natural modern home based on the following information: Style: Natural Modern, Layout: 4LDK, Budget: 50 million yen, Materials used: Wood, glass, concrete" is passed to the generative AI model.

[1187] The user can view the generated 3D model in VR space, and if they feel there is a problem with the living space, they can input a correction instruction such as "make the living space larger." The server receives this correction instruction and updates the 3D model again using the generated AI model. This process is repeated until a model that satisfies the user is finally completed. Once the user confirms the final order, the construction project begins, and progress information is notified to the user in real time.

[1188] This system allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[1189] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1190] Step 1:

[1191] The user launches the smartphone app and inputs the desired housing characteristics (e.g., style, layout, budget, materials used, etc.). The input data is initially stored on the user's device and later sent to the server.

[1192] Input: House characteristic data entered by the user.

[1193] Output: House feature data temporarily stored on the user's device.

[1194] Step 2:

[1195] The device sends the user's input data to the server, which then analyzes the data. This analysis includes data validation and missing value completion.

[1196] Input: House characteristic data sent from the user terminal.

[1197] Output: Analyzed housing data stored on the server.

[1198] Step 3:

[1199] The server generates an initial 3D model using a generative AI model based on the analyzed data. A prompt is generated and input into the generative AI to generate the 3D model.

[1200] Input: Parsed housing data, prompt for the generative AI model.

[1201] Output: The generated 3D model data.

[1202] Step 4:

[1203] The server sends the generated 3D model data to the user's device, where the 3D model is displayed and the user can confirm the model.

[1204] Input: Generated 3D model data.

[1205] Output: The 3D model displayed on the user's device.

[1206] Step 5:

[1207] The user checks the 3D model and inputs correction instructions as necessary. The correction instructions are entered into the terminal as specific requests (e.g., "Make the living room bigger") and sent to the server.

[1208] Input: Correction instructions entered by the user.

[1209] Output: Correction instruction data sent to the server.

[1210] Step 6:

[1211] The server regenerates the 3D model based on the received correction instructions, and then passes the prompt sentence to the generative AI model again to generate an updated 3D model.

[1212] Input: User correction instruction data, update prompt sentence to the generative AI model.

[1213] Output: Updated 3D model data.

[1214] Step 7:

[1215] The server then sends the updated 3D model back to the user's device, where the user reviews the new 3D model and repeats this process until the modifications are complete.

[1216] Input: Updated 3D model data.

[1217] Output: The updated 3D model displayed on the user's device.

[1218] Step 8:

[1219] When the user finally completes a model that satisfies them, they press the order confirmation button, and the terminal sends the confirmed order data to the server.

[1220] Input: Final confirmed 3D model, order confirmation information.

[1221] Output: Order confirmation data sent to the server.

[1222] Step 9:

[1223] The server sends the order information to the construction company based on the received order confirmation data. It also starts managing the construction project, periodically receives progress information from the construction company, and notifies the user terminal.

[1224] Input: Order confirmation data, project progress information.

[1225] Output: Order information sent to the construction company, progress information notified to the user terminal.

[1226] Step 10:

[1227] The 3D model is displayed in a VR space so that users can take a virtual tour through their devices. Users can connect their smartphone to a VR headset and enjoy the virtual tour experience.

[1228] Input: 3D model data to be displayed in VR space.

[1229] Output: 3D model displayed in a VR headset for virtual tours.

[1230] Through these steps, users can design their desired home in detail, check, modify, and finalize the design in virtual space, and keep track of the project's progress in real time.

[1231] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1232] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[1233] User requirement collection and emotion recognition

[1234] A user downloads and installs the ordering app. When the user launches the app, a screen for entering basic information appears. The ordering app has the ability to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions during the process.

[1235] Generating 3D models and reflecting emotion data

[1236] The device sends the user's input data and the recognized emotion data to the server. The server then first analyzes the characteristic data of the user's desired home, and then analyzes the emotion data. Based on the analysis results, the generative AI then activates and generates an initial 3D model that takes the user's emotion data into account. This process makes it possible, for example, to design a home that incorporates elements that the user feels positive about.

[1237] Emotional reflection of 3D model checking and correction

[1238] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs corrections they would like to make to the model, their emotions are also collected at the same time. For example, along with instructions such as "I would like the living space to be a little larger," the user's emotions at that time (dissatisfaction, expectations, etc.) are sent.

[1239] The device sends these correction instructions and emotion data to the server, which then uses the generative AI to correct the 3D model. The emotion data is used to predict the importance of the user's request and makes optimal corrections based on that.

[1240] Final order confirmation

[1241] When the user is satisfied with the final model, they press the order confirmation button. The device then sends the final 3D model data, order information, and emotion data to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is then sent to the device as a notification to the user.

[1242] Emotional reflection in project management and progress notifications

[1243] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at the set timing. The emotion engine also operates when notifications are sent, capturing the user's emotions when providing feedback and sending them to the server.

[1244] Specific examples

[1245] Example 1:

[1246] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[1247] Example 2:

[1248] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[1249] In this way, by incorporating an emotion engine, the system of the present invention can design a house that reflects the user's feelings, thereby achieving higher customer satisfaction.

[1250] The processing flow will be explained below.

[1251] Step 1:

[1252] A user downloads and installs the ordering app. When the user launches the app, a screen appears where the user can enter their information (name, contact details).

[1253] Step 2:

[1254] In addition to a screen for entering basic information, the device also activates an emotion engine that captures the user's facial expressions and voice. As the user enters information, the emotion engine captures the user's emotional data in real time.

[1255] Step 3:

[1256] The user inputs the home's characteristics (style, layout, budget, materials used, etc.) into the ordering app. The emotion engine captures the user's emotions while they are typing.

[1257] Step 4:

[1258] The device temporarily stores the user's input data and emotion data in a local database and then transmits it to the server.

[1259] Step 5:

[1260] The server receives and analyzes the user's input data, then analyzes the emotion data. Based on these analysis results, a generative AI is activated to generate an initial 3D model that takes the user's emotions into account.

[1261] Step 6:

[1262] The server sends the generated 3D model and the emotion analysis results to the device, which then displays the 3D model to the user and asks for their confirmation.

[1263] Step 7:

[1264] The user checks the 3D model and inputs the parts they want to modify. At the same time, the emotion engine captures the user's emotions and sends them, along with the modification instructions, from the device to the server.

[1265] Step 8:

[1266] The server receives the modification instructions and emotion data, and again modifies the 3D model using the generative AI. At this time, the model is adjusted with an emphasis on the user's emotion data. For example, it focuses on improving areas that the user expressed dissatisfaction with.

[1267] Step 9:

[1268] The server resends the modified 3D model to the device, which displays it to the user and asks for confirmation again. This process is repeated until the user is satisfied.

[1269] Step 10:

[1270] If the user is satisfied with the final model, they press the "Confirm Order" button, and the device sends the final 3D model data, order information, and emotion data to the server.

[1271] Step 11:

[1272] The server collects the order details and sends the order data to the affiliated construction company. This confirmation information is then sent to the terminal and notified to the user.

[1273] Step 12:

[1274] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[1275] Step 13:

[1276] The server periodically sends progress information to the device. The device notifies the user of the progress and displays it. The emotion engine also works when notifying the user, capturing the emotions expressed by the user regarding the progress.

[1277] Step 14:

[1278] Users can check the progress and enter feedback as needed. For example, requests such as "I want to change the color of the exterior walls" are captured along with their emotions at the time.

[1279] Step 15:

[1280] The device sends the user's feedback and emotional data to a server, which analyzes the feedback and forwards it to the construction company.

[1281] Step 16:

[1282] The construction company makes corrections based on the feedback and performs a final check. The server sends a completion notification to the terminal and notifies the user of the date and time of handover. The user makes a final check and the handover is completed in a satisfactory manner.

[1283] Example 2

[1284] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1285] Conventional home design systems can collect specific user requirements, but do not consider the emotions felt by the user during the design process. As a result, the user's emotions and satisfaction are not reflected in the design process, which can lead to dissatisfaction with the final home design. In particular, there is a need for a system that can accurately reflect the user's emotions when making changes to the layout or style of the home.

[1286] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1287] In this invention, the server includes: a means for a user to input the characteristics of a home desired by the user; a means for the terminal to capture the user's facial expressions and voice in real time and recognize emotions; a means for the server to receive and analyze the user's input data and emotional data; a means for the server to generate a 3D model using a generation AI based on the analyzed data; a means for the server to send the generated 3D model data to the terminal; a means for the user to check the 3D model and input correction instructions; a means for the terminal to recapture the user's facial expressions and voice and recognize emotions; a means for the server to receive the correction instructions and emotional data and regenerate the 3D model; a means for the user to make a final confirmation and confirm the order; a means for the server to send the order data to a construction company and manage the project; and a means for the server to notify the terminal of project progress information. This makes it possible to reflect the user's emotions in the home design process and provide a final home design that will provide a high level of satisfaction.

[1288] "User" refers to an individual or corporation that uses the system to design a desired home.

[1289] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[1290] "Facial and audio capture" refers to the process by which the device records the user's facial movements and voice in real time.

[1291] "Emotion recognition" refers to analyzing a user's emotional state (e.g., joy, sadness, anticipation, dissatisfaction) from captured facial and voice data.

[1292] "Server" refers to a central computing device that processes and manages data for the entire system.

[1293] "Input data" refers to information including the characteristics and requirements of the home desired by the user.

[1294] "Emotion data" refers to information about emotions recognized from the user's facial expressions and voice.

[1295] "Analyzing" refers to the server processing the received data with an analysis algorithm.

[1296] "Generative AI" refers to a system that uses artificial intelligence technology to generate a 3D model of a house based on user requirements and emotional data.

[1297] "3D model" refers to three-dimensional design data for a house that is generated based on the user's wishes and emotional data.

[1298] "Sending" refers to the process by which the server transfers the generated data to the terminal.

[1299] "Modification instructions" refer to requests for changes or adjustments made by the user to the 3D model.

[1300] "Regenerate" refers to the server recreating the 3D model based on the correction instructions and emotion data.

[1301] "Final confirmation" refers to when the user is satisfied with the final 3D model and confirms the order.

[1302] "Order Data" means all order information included with the final confirmed 3D model.

[1303] "Construction company" refers to the corporation that actually constructs the housing ordered.

[1304] "Managing the project" refers to the server monitoring and managing the progress of the entire home building process.

[1305] "Progress Information" means data relating to the progress of a construction project.

[1306] "Notifying" refers to the process by which the server communicates progress and other important information to the device.

[1307] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[1308] User requirement collection and emotion recognition

[1309] A user downloads and installs a home design app on their smartphone or tablet. When the app is launched, a screen for entering basic information appears. The app is equipped with a function to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions in real time. Specifically, the app uses the smartphone's camera and microphone to acquire emotion data using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[1310] Generating 3D models and reflecting emotion data

[1311] The device sends the user's input data and emotion data to the server. Upon receiving this data, the server first analyzes the user's request data, and then analyzes the emotion data. Based on the analysis results, the server activates the generative AI to generate an initial 3D model that reflects the user's emotion data. The server then sends the generated 3D model data to the device, and uses an interactive viewer (e.g., Three.js) to display the 3D model on the smartphone screen.

[1312] Emotional reflection of 3D model checking and correction

[1313] When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs the parts of the model they want to modify, their emotions are also collected at the same time. Specifically, the camera and microphone are used again to collect the user's emotional data. The device sends these modification instructions and emotional data to the server, and the server again uses the generative AI to modify the 3D model. The emotional data is used to determine the importance of the user's request, and the optimal modifications are made based on that.

[1314] Final order confirmation and project management

[1315] When the user is satisfied with the final model, they press the order confirmation button. The device sends the final 3D model data, order information, and emotion data to the server. The server consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user. Once the project begins, the server manages the progress of the construction project. The construction company periodically reports progress information to the server, and the server receives this progress information and notifies the device at the set time. The emotion engine also runs when notifying, capturing the emotion of the user's feedback and sending it to the server.

[1316] Examples and thought-provoking prompts

[1317] Example 1

[1318] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[1319] Prompt Sentence Examples

[1320] If a user looking for a natural modern 4LDK home expresses dissatisfaction with the size of the living space, please suggest how to improve it based on emotional data.

[1321] Example 2

[1322] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[1323] Prompt Sentence Examples

[1324] If a user with a budget constraint requests to change the location of the bedroom in a minimalist 2LDK apartment, please suggest the optimal layout based on the sentiment data.

[1325] In this way, by incorporating an emotion engine, the system of the present invention can design a home that reflects the user's feelings, achieving higher customer satisfaction. The hardware used to implement this system includes a smartphone and a server, and specific software includes facial recognition software (e.g., OpenFace), voice analysis software (e.g., IBM Watson), and an interactive viewer (e.g., Three.js).

[1326] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1327] Step 1:

[1328] A user downloads and installs a home design app.

[1329] What it does: Download and install the app from the app store on your smartphone or tablet.

[1330] Input: None

[1331] Output: Device with home design app installed

[1332] Step 2:

[1333] The user launches the app and is presented with a screen for entering basic information.

[1334] Specific operation: When the user taps the app icon to launch the app, a screen for entering basic information appears on the device.

[1335] Input: None

[1336] Output: Basic information input screen

[1337] Step 3:

[1338] Users enter details such as their name, contact information, desired home style and layout, budget, and materials used.

[1339] Specific behavior: The user enters the required information into the form and proceeds to the next screen.

[1340] Input: User-entered house characteristics data

[1341] Output: The entered house feature data is saved on the device.

[1342] Step 4:

[1343] The device captures the user's facial expressions and voice in real time and recognizes their emotions.

[1344] Specific operation: The smartphone's camera and microphone are activated, and emotion data is obtained using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[1345] Input: User's facial expression and voice data

[1346] Output: Emotion data

[1347] Step 5:

[1348] The terminal transmits the user's input data and emotion data to the server.

[1349] Specific behavior: The input data and captured emotion data are sent to the server using the HTTPS protocol.

[1350] Input: House feature data, emotion data

[1351] Output: Data sent to the server

[1352] Step 6:

[1353] The server analyzes the received data and generates an initial 3D model using a generative AI model.

[1354] Specific operation: The server analyzes the received data and obtains the analysis results using a deep learning model (e.g., TensorFlow). Based on that data, it uses generative AI (e.g., Unity3D or Blender) to render a 3D model.

[1355] Input: House feature data, emotion data

[1356] Output: Initial 3D model

[1357] Step 7:

[1358] The server sends the generated 3D model data to the terminal.

[1359] Specific operation: The generated 3D model data is encoded and sent to the terminal using the HTTPS protocol.

[1360] Input: Initial 3D model

[1361] Output: 3D model sent to device

[1362] Step 8:

[1363] The device displays the 3D model to the user.

[1364] Specific operation: Use an interactive viewer (e.g., Three.js) to display a 3D model on the smartphone screen and allow the user to check it.

[1365] Input: 3D model data

[1366] Output: A displayed 3D model

[1367] Step 9:

[1368] The user checks the 3D model and inputs correction instructions.

[1369] Specific operation: The user adjusts the model using the touch interface of the smartphone.

[1370] Input: User correction instructions

[1371] Output: Correction instruction data

[1372] Step 10:

[1373] The device recaptures the user's facial expressions and voice to recognize emotions.

[1374] Specific operation: Again, the camera and microphone are used to collect user emotion data.

[1375] Input: User's facial expression and voice data

[1376] Output: Emotion data at the time of correction

[1377] Step 11:

[1378] The device sends correction instructions and emotion data to the server.

[1379] Specific operation: Input data and emotion data are sent to the server using the HTTPS protocol.

[1380] Input: Correction instruction data, emotion data when correcting

[1381] Output: Correction data and emotion data sent to the server

[1382] Step 12:

[1383] The server regenerates the 3D model based on the correction instructions and emotion data.

[1384] Specific operation: The server integrates the correction instructions and emotion data, and again uses the generative AI to generate a corrected 3D model.

[1385] Input: Correction instruction data, emotion data when correcting

[1386] Output: Modified 3D model

[1387] Step 13:

[1388] If the user is satisfied with the final model, he or she presses the order confirmation button.

[1389] Specific operation: The user presses the order confirmation button on the smartphone.

[1390] Input: User order confirmation operation

[1391] Output: Order confirmation data

[1392] Step 14:

[1393] The terminal sends the final 3D model data, order information, and emotion data to the server.

[1394] Specific behavior: The final data is sent to the server using the HTTPS protocol.

[1395] Input: Final 3D model data, order information, emotion data

[1396] Output: Final data sent to the server

[1397] Step 15:

[1398] The server collects the order information and sends the order data to the affiliated construction company.

[1399] Specific operation: The server sends the order data to the construction company via email or API.

[1400] Input: Order details data

[1401] Output: Order data sent to construction company

[1402] Step 16:

[1403] The server notifies the user of the order confirmation information.

[1404] Specific operation: The server sends a push notification to the device to notify the user that the order has been confirmed.

[1405] Input: Order confirmation information

[1406] Output: Push notification to the user

[1407] Step 17:

[1408] The server manages the progress of the construction project.

[1409] What it does: The server uses project management software (e.g. Jira) to track progress.

[1410] Input: Project Management Information

[1411] Output: Managed progress data

[1412] Step 18:

[1413] The construction company reports progress information to the server.

[1414] What happens: A construction company uses a progress reporting tool (e.g., Microsoft Project) to send progress data to a server.

[1415] Input: Progress information

[1416] Output: Progress data sent to the server

[1417] Step 19:

[1418] The server notifies the user of progress information.

[1419] Specific operation: The server sends a push notification to the device at the set timing.

[1420] Input: Progress information

[1421] Output: Progress notification to user

[1422] Step 20:

[1423] The device will display a notification and the emotion engine will capture the emotion during the feedback.

[1424] Specific behavior: When the user confirms the notification, the device again uses the camera and microphone to capture emotion data.

[1425] Input: Emotional data as user feedback

[1426] Output: Feedback emotion data sent to the server

[1427] (Application example 2)

[1428] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1429] The current home design process rarely takes into account user emotions, making it difficult to increase true user satisfaction. Virtual stores also lack personalized responses based on user emotions, creating a need for improved user experience. Conventional technologies lack the means to analyze user emotions in real time and customize products and services based on those emotions, making user satisfaction a challenge.

[1430] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input the characteristics of the house desired by the user; means for the terminal to capture the user's facial expressions and voice and acquire emotional data; means for the server to receive and analyze the user's input data and emotional data; means for the server to generate a 3D model based on the analyzed data and taking the emotional data into consideration using a generation AI; means for the server to send the generated 3D model data to the user terminal; means for the user to check the 3D model and input correction instructions; means for the terminal to re-capture the user's facial expressions and voice and acquire emotional data; means for the server to receive the correction instructions and emotional data and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to send the order data to the construction company and manage the project; and means for the server to notify the user terminal of project progress information. This makes it possible to provide more personalized products and services that reflect the user's emotions in real time.

[1431] The "means for the user to input the characteristics of the home desired" is a device or system that provides an interface for the user to specifically input the requirements and desires regarding the ideal home.

[1432] "Means for the device to capture the user's facial expressions and voice and obtain emotional data" refers to technology that captures the user's facial expressions and voice in real time, analyzes them, and obtains the user's emotions as data.

[1433] The "means for the server to receive and analyze the input data and emotion data of the user" refers to a device or system that receives data related to housing requirements and emotions sent from the user and performs an analysis process.

[1434] "Means for the server to use generative AI based on the analyzed data to generate a 3D model that also takes into account emotional data" refers to a technology that uses generative AI to create a 3D model that reflects the user's emotions based on analyzed data.

[1435] "Means by which the server transmits the generated 3D model data to the user terminal" refers to a mechanism for transferring the generated 3D model data to the terminal operated by the user.

[1436] "Means for the user to check the 3D model and input correction instructions" refers to a device or system that displays the 3D model to the user and provides an interface that allows the user to check it and then input correction instructions.

[1437] "Means for the device to recapture the user's facial expressions and voice and obtain emotional data" refers to technology that recaptures the user's facial expressions and voice and digitizes their emotions when checking and correcting the 3D model.

[1438] "Means for the server to receive correction instructions and emotion data and regenerate the 3D model" refers to technology that regenerates the 3D model based on correction instructions sent by the user and re-acquired emotion data.

[1439] "Means for the user to make final confirmation and confirm the order" refers to the procedures and devices by which the user can finally confirm the 3D model and officially confirm the order after being satisfied with it.

[1440] The "means for the server to transmit order data to the construction company and manage the project" is a system for transmitting confirmed order data to the construction company and managing the progress of the project.

[1441] "Means for the server to notify the user terminal of project progress information" refers to a technology that periodically notifies the terminal operated by the user of the progress status of the project.

[1442] The present invention is a system that recognizes a user's emotions and reflects them in the home design process, enabling more satisfying home designs. It can also be applied as a system for suggesting products in a virtual store based on the user's emotions. Detailed embodiments of the system are described below.

[1443] System Program

[1444] This system recognizes the user's emotions and analyzes their emotional data to suggest the most suitable 3D models and products for the user. This process includes the following elements:

[1445] 1. Obtaining user input and emotion data:

[1446] Users download a dedicated application and input the desired characteristics for their home design. The camera and microphone on their device (such as a smartphone or tablet) capture the user's facial expressions and voice in real time to obtain emotional data. The DeepFace library is used to analyze facial expressions and recognize key emotions (e.g., "happiness," "sadness," "surprise," etc.).

[1447] 2. Data Receipt and Analysis:

[1448] The input data and emotion data acquired by the device are sent to a server, where they are simultaneously analyzed and home design data that reflects the user's emotions is generated. This analysis is performed using machine learning models and generative AI.

[1449] 3. Generate 3D model:

[1450] Using generative AI, an initial 3D model is generated that takes into account the user's emotional data. This generation takes into account the user's desired home characteristics data and the results of emotional analysis.

[1451] 4. Check and correct the 3D model:

[1452] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model and inputs correction instructions or desired changes, the device again captures facial expressions and voice to obtain emotional data. This allows the device to analyze the importance of the correction instructions and the intensity of the emotion, allowing for more appropriate corrections.

[1453] 5. Final model confirmation and project management:

[1454] Once the user is satisfied with the final 3D model, they confirm their order. The server sends the order data to the construction company and manages the project progress. Project progress information is periodically sent to the user's device, and emotional data is also collected during the project.

[1455] Hardware and software used

[1456] Hardware:

[1457] Smartphone, tablet, or PC camera

[1458] microphone

[1459] software:

[1460] OpenCV

[1461] DeepFace

[1462] Hugging Face transformers library

[1463] Specific examples

[1464] 1. User's house design:

[1465] A user looking for a natural modern-style home inputs their information into the application. As they input, the camera and microphone capture their facial expressions and voice, and the emotion engine recognizes that the user is expressing strong expectations for the living space. Based on this, the generative AI creates an initial 3D model and presents it to the user.

[1466] Example prompt sentence:

[1467] "Users are expressing joy. Please suggest appropriate living space layouts."

[1468] 2. Product suggestions in virtual stores:

[1469] The facial expressions and voices of users shopping using a virtual store application are captured, and if the user expresses feelings of fatigue, the generative AI will suggest relaxing products (e.g., aroma candles).

[1470] Example prompt sentence:

[1471] "The user is showing signs of fatigue. Please suggest appropriate relaxation items."

[1472] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1473] Step 1:

[1474] A user downloads and installs a home design app. When the app is launched, a screen for entering basic information appears. While the user enters the desired home characteristics (style, layout, budget, materials used, etc.), the device's (smartphone or tablet) camera and microphone capture the user's facial expressions and voice in real time, obtaining emotion data. This data is obtained using the OpenCV and DeepFace libraries. The input is the user's information, facial expressions, and voice data, and the output is emotion data.

[1475] Step 2:

[1476] The input data (house features) and emotion data acquired by the device are sent to the server. The server receives the sent data and performs an analysis process. The server analyzes the house data entered by the user and the captured emotion data to identify the house features desired by the user. The input is the house features and emotion data, and the output is the analyzed feature data.

[1477] Step 3:

[1478] The server uses a generative AI based on the analyzed data to generate an initial 3D model that also takes into account emotional data. The Hugging Face generative AI model is used for this generation. The analysis results and a prompt are input into the generative AI model, which outputs a 3D model that reflects the user's emotions. The input is the analyzed feature data, emotional data, and the prompt "Please propose the layout of the living space that the user desires," and the output is the initial 3D model.

[1479] Step 4:

[1480] The server sends the generated 3D model data to the user's device. The user checks the 3D model on the device and inputs any complaints or corrections. At this time, the device again captures the user's facial expressions and voice, and obtains emotional data accompanying the corrections. The input is the 3D model data, and the output is the corrections and emotional data.

[1481] Step 5:

[1482] The device sends the correction instructions and emotion data to the server. The server receives them and regenerates the 3D model using the analysis and generative AI. The input is the correction instructions, emotion data, and the prompt "Please suggest the optimal layout of the living space based on the user's correction instructions." The output is the corrected 3D model.

[1483] Step 6:

[1484] Once the user has finalized the 3D model that satisfies them, they confirm their order. The terminal sends the final model data and order information to the server. The input is the confirmed 3D model and order information, and the output is the final order data.

[1485] Step 7:

[1486] The server sends order data to the construction company and manages the progress of the project. Progress information on ongoing projects is periodically reported to the server, which then notifies the user's device. When notifying the user, the device also captures the user's facial expression and collects emotional data. The input is progress information from the construction company, and the output is progress notification and emotional data to the user.

[1487] This series of processes enables the design of homes that reflect the user's emotions and provide a higher level of satisfaction. Furthermore, emotion-based product suggestions can be made as needed.

[1488] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1489] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1490] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1491] [Fourth embodiment]

[1492] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1493] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1494] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1495] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1496] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1497] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1498] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1499] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1500] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1501] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1502] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1503] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1504] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1505] The present invention is a system for efficiently designing and quickly providing a home that a user desires. This system inputs the characteristics of the home that the user desires, analyzes that information, and generates a 3D model of the home. Details of the system and its embodiments are described below.

[1506] User requirements collection

[1507] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen, where they enter details such as their name, contact information, desired home style (e.g., natural modern, minimalist, classic, etc.), floor plan, budget, and materials used.

[1508] 3D model generation and modification

[1509] The device temporarily stores the information entered by the user in a local database and then sends it to the server. The server then analyzes the data and uses generative AI to generate an initial 3D model based on the user's requests.

[1510] The generated 3D model is sent to the device and displayed to the user. The user reviews the 3D model and indicates any necessary modifications. For example, they can input specific requests such as "I want to make the living space larger" or "I want to change the location of the kitchen." The device then sends the modifications back to the server, which then regenerates the 3D model based on those instructions. This modification process is repeated until the user is satisfied.

[1511] Final order confirmation

[1512] When the user is satisfied with the final model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[1513] Project management and progress notifications

[1514] Once the order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check the progress status through the app, such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color."

[1515] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[1516] Specific examples

[1517] Example 1:

[1518] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[1519] Example 2:

[1520] Another user requests a minimalist 2LDK layout. They set their budget and materials in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests a change in the location of the bedroom. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[1521] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[1522] The processing flow will be explained below.

[1523] Step 1:

[1524] The user downloads and installs the ordering app. The user launches the app and enters their user information (name, contact details).

[1525] Step 2:

[1526] The terminal displays a form for the user to input the characteristics of the home they desire (style, layout, budget, materials, etc.). The user enters this information.

[1527] Step 3:

[1528] The terminal temporarily stores the user's input data in a local database and then transmits it to the server.

[1529] Step 4:

[1530] The server receives and analyzes the user's input data. Based on the analysis results, it activates the generation AI to generate the initial 3D model.

[1531] Step 5:

[1532] The server sends the generated 3D model to the device, which displays the 3D model to the user and asks for confirmation.

[1533] Step 6:

[1534] The user checks the 3D model and inputs any modifications they would like to make, such as making the living space larger.

[1535] Step 7:

[1536] The device sends the user's correction instructions back to the server, which receives the instructions, uses the generation AI to correct the 3D model again, and sends it back to the device.

[1537] Step 8:

[1538] This process is repeated several times until a final model that satisfies the user is determined, and the user presses the "Confirm Order" button.

[1539] Step 9:

[1540] The terminal sends the final 3D model data and order information to the server, which then aggregates the order details and sends the order data to the construction company.

[1541] Step 10:

[1542] The server sends a notification of order confirmation to the terminal, which receives the notification and displays it to the user.

[1543] Step 11:

[1544] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[1545] Step 12:

[1546] The server periodically sends progress information to the terminal, which then notifies and displays the progress information to the user.

[1547] Step 13:

[1548] The user can check the progress and enter feedback as needed, for example, a request such as "I would like to change the color of the exterior walls."

[1549] Step 14:

[1550] The device sends the feedback to the server, which forwards it to the construction company.

[1551] Step 15:

[1552] The construction company makes corrections based on the feedback and performs a final check. The server notifies the user of completion and the date and time of handover.

[1553] Step 16:

[1554] The user makes a final check and the delivery is completed in a satisfactory manner.

[1555] Example 1

[1556] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1557] The traditional residential design process required a great deal of time and effort to meet the detailed needs of the user, making it difficult to design efficiently. Furthermore, there was a lack of collaboration and communication between the user and the construction company, making it difficult to make design changes or check progress. This could lead to lower customer satisfaction and project delays.

[1558] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1559] In this invention, the server includes: means for a user to input the characteristics of a home desired by the user; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model using a generative AI based on the analyzed data; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; means for the user to input order and correction data through an application; means for the server to operate the generative AI model using prompt sentences based on the user's input data; and means for the terminal to transmit order confirmation information from the user to the server. This enables home design that can quickly and efficiently meet the user's detailed needs, facilitating project progress management and communication with users.

[1560] A "user" is an individual or corporation that wishes to design and order a home.

[1561] An "ordering app" is a software application that allows users to enter the information necessary to design and order a home.

[1562] The "server" is a computer system that has the function of analyzing data collected from users, generating and modifying 3D models using generative AI, and finally transmitting order data to construction companies.

[1563] "Terminal" refers to a device that allows a user to input information via the ordering app, check the 3D model, and input correction instructions.

[1564] A "construction company" is a company or organization that actually constructs a house based on the order data sent from the server.

[1565] "Generative AI" is an artificial intelligence technology that automatically generates and modifies 3D models of homes based on user input data.

[1566] "3D Model" means a three-dimensional digital model of a house generated based on a user's requirements.

[1567] "Modification instructions" are input information for changes or adjustments that the user makes to the 3D model after checking it.

[1568] A "prompt" is a textual instruction used by the server to operate a generative AI model.

[1569] "Project progress information" is data that indicates the status and progress of housing projects currently underway by construction companies.

[1570] "Feedback" refers to additional requests or change requests made by users during the progress of a construction project.

[1571] "Confirmed order information" is the final confirmation data that is sent when the user is satisfied with the final model and officially orders the house.

[1572] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[1573] User requirements collection

[1574] First, users download and install the ordering app. When they launch the app, they are presented with a basic information entry screen where they enter details such as their name, contact information, desired home style (natural modern, minimalist, classic, etc.), floor plan, budget, and materials used. Information is entered using a device such as a smartphone or tablet.

[1575] 3D model generation and modification

[1576] The device temporarily stores the information entered by the user in a local database and then sends it to a server. The server used here is, for example, a cloud-based server system. The server that receives the data first analyzes the data and then uses a generative AI (such as OpenAI's DALL-E or Autodesk's Revit) to generate an initial 3D model. This 3D model is generated based on the user's requirements.

[1577] The generated 3D model is sent to the device and displayed to the user. The user checks the 3D model and, if any modifications are needed, inputs specific instructions. For example, instructions might be to "make the living space larger" or "change the location of the kitchen." The device then sends these instructions back to the server.

[1578] The server regenerates the 3D model based on the modification instructions, again using the generated AI model and prompts to make specific modifications to the model. The modified 3D model is then sent back to the user's device, and the process is repeated until the user is satisfied.

[1579] Final order confirmation

[1580] When the user is satisfied with the final 3D model, they press the order confirmation button. At that point, the device sends the final 3D model data and order information to the server. The server then consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user.

[1581] Project management and progress notifications

[1582] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at set times. Users can check progress status such as "Foundation work has been completed" or "We are now starting to finalize the exterior wall color" through the app.

[1583] Furthermore, if users have any requests for changes or additions along the way, they can send feedback via their devices. The server analyzes the feedback and forwards it to the construction company. This two-way communication significantly improves project transparency and customer satisfaction.

[1584] Specific examples

[1585] Example 1:

[1586] A user requests a natural modern-style home with a 4LDK floor plan. They input the necessary information through the app, and the server uses generative AI to provide the user with an initial 3D model. The user is concerned about the size of the living space, so they input a correction request to "make the living space larger." The server generates a new 3D model, which the user can review. This process is repeated several times until a satisfactory model is finally completed. The user then confirms the order and the project begins.

[1587] Example 2:

[1588] Another user requests a minimalist two-bedroom apartment, specifying their budget and materials to be used in detail through the app. The server uses generative AI to create a 3D model based on this information and sends it to the device. The user reviews the model and requests changes to the bedroom location. If satisfied with the revised model, they confirm the order and the construction project begins. The user is periodically notified of construction progress, and can check the progress at any time.

[1589] In this way, by using the system of the present invention, users can efficiently design and quickly provide homes that meet their individual needs.

[1590] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1591] Step 1:

[1592] The user downloads and installs the ordering app. The user launches the app and enters details such as name, contact information, desired home style, layout, budget, and materials on the basic information input screen.

[1593] Input: User information (name, contact details, house style, layout, budget, materials used, etc.)

[1594] Output: User information entered

[1595] Specific actions: The user operates a smartphone or tablet, fills in the required information in the information input form, and presses the submit button.

[1596] Step 2:

[1597] The terminal temporarily stores the entered user information in a local database and then transmits the information to the server.

[1598] Input: User information (obtained from local database)

[1599] Output: Data sent to the server (user information)

[1600] Specific operation: The device sends user information to the server using an HTTP request.

[1601] Step 3:

[1602] The server analyzes the received user information and generates an initial 3D model using generative AI (e.g., OpenAI's DALL-E or Autodesk's Revit).

[1603] Input: User information (analyzed by the server)

[1604] Output: Initial 3D model data

[1605] Specific operation: The server parses the user information and asks the generation AI to generate a 3D model using a prompt, such as "Please generate a 3D model of a natural modern style 4LDK house."

[1606] Step 4:

[1607] The server sends the generated 3D model data to the terminal and displays it to the user.

[1608] Input: Initial 3D model data

[1609] Output: Data sent to the device (initial 3D model)

[1610] Specific operation: The server returns the generated 3D model data to the terminal, and the terminal displays the 3D model to the user.

[1611] Step 5:

[1612] The user checks the 3D model displayed on the device and inputs any necessary corrections, such as "make the living space larger" or "change the location of the kitchen."

[1613] Input: Initial 3D model, user instructions for modification

[1614] Output: Correction instruction data

[1615] Specific action: The user enters the specific changes in text format into the correction instruction input form of the app and presses the submit button.

[1616] Step 6:

[1617] The terminal transmits a correction instruction to the server.

[1618] Input: Correction instruction data (obtained from the terminal)

[1619] Output: Data to be sent to the server (correction instructions)

[1620] Specific operation: The device sends correction instructions to the server using an HTTP request.

[1621] Step 7:

[1622] The server regenerates the 3D model based on the received modification instructions, and again uses a generation AI (such as OpenAI's DALL-E or Autodesk's Revit) to modify the model using prompts.

[1623] Input: Correction instruction data

[1624] Output: Regenerated 3D model data

[1625] Specific operation: The server analyzes the modification instructions and sends a regeneration prompt to the generation AI, such as "Please regenerate a 3D model of a 4LDK natural modern style house with an expanded living space."

[1626] Step 8:

[1627] The server transmits the regenerated 3D model data to the terminal and displays it to the user.

[1628] Input: Regenerated 3D model data

[1629] Output: Data sent to the device (corrected 3D model)

[1630] Specific operation: The server sends the regenerated 3D model to the device, which then displays it to the user. This process is repeated until the user is satisfied.

[1631] Step 9:

[1632] If the user is satisfied with the final model, he or she presses the order confirmation button.

[1633] Input: Final 3D model, order confirmation instructions

[1634] Output: Confirmed order data

[1635] Specific operation: The user presses the order confirmation button, and the terminal sends the confirmation data to the server.

[1636] Step 10:

[1637] The terminal transmits the final 3D model data and order information to the server.

[1638] Input: Confirmed order data

[1639] Output: Data sent to the server (final 3D model data, order information)

[1640] What happens: The device sends the final 3D model and order data to the server using an HTTP request.

[1641] Step 11:

[1642] The server transmits the received order data to affiliated construction companies and manages the project.

[1643] Input: Order data

[1644] Output: Data to be sent to construction companies (order data)

[1645] Specific operation: The server sends the order data to the construction company and registers it in the project management system.

[1646] Step 12:

[1647] The server notifies the user's terminal of the progress information of the project received from the construction company.

[1648] Input: Progress information (received from construction company)

[1649] Output: Notification data (progress information) sent to the user's device

[1650] Specific operation: The server periodically checks progress information and sends notifications to the user's device at appropriate times.

[1651] This allows users to move through the home design process efficiently and quickly, facilitating overall communication and management right up to the final order.

[1652] (Application example 1)

[1653] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1654] There is a demand for efficient design of the homes users desire and a highly reproducible virtual viewing experience. However, conventional systems were unable to quickly generate and modify 3D models that matched the user's wishes, and lacked a means to check the progress in real time. In addition, users could not actually view the home before it was completed, which could lead to low satisfaction with the final home design.

[1655] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1656] In this invention, the server includes: means for a user to input the characteristics of a desired home; means for the server to receive and analyze the user's input data; means for the server to generate a 3D model based on the analysis data using a generation AI; means for the server to transmit the generated 3D model data to a user terminal; means for the user to check the 3D model and input correction instructions; means for the server to receive the correction instructions and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to transmit the order data to a construction company and manage the project; means for the server to notify the user terminal of project progress information; and means for displaying the 3D model in a VR space on the user terminal, allowing the user to virtually tour the home. This allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[1657] A "user terminal" is a device used by a user to input the characteristics of a house, check the 3D model, input correction instructions, and take a virtual tour.

[1658] The "server" is a device or system that receives and analyzes data sent from user devices and generates 3D models using generative AI. It also manages project progress information and notifies user devices.

[1659] "Generative AI" is an artificial intelligence model that automatically generates a 3D model based on the characteristic data of a house entered by the user.

[1660] A "3D model" is a digital model that represents the shape, layout, style, etc. of the home desired by the user in three dimensions.

[1661] "VR space" is a virtual space that uses virtual reality technology to allow users to tour homes in a virtual environment.

[1662] "Modification instructions" are input information that a user can use to request changes or improvements to specific parts of a 3D model after checking it.

[1663] A "virtual tour" is a virtual home tour method that allows users to use a VR space to actually walk around the generated 3D model and experience it for themselves.

[1664] "Progress information" is data that indicates the progress of construction and important events in a user's housing project.

[1665] MODE FOR CARRYING OUT THE INVENTION

[1666] This invention is a system for efficiently designing and quickly delivering the home a user desires. The system inputs the characteristics of the home a user desires, analyzes that information, and generates a 3D model of the home.

[1667] User requirements collection

[1668] First, users install a dedicated application on their smartphone. When they launch the application, a screen appears where they can enter details such as the desired style of the home (e.g., natural modern, classic, minimalist, etc.), floor plan, budget, and materials used. This data is temporarily stored on the user's device and then sent to the server.

[1669] 3D model generation and modification

[1670] The server receives and analyzes the data sent by the user. Based on the analysis results, the server uses a generative AI model to generate an initial 3D model. After generation, this 3D model is sent to the user's device and displayed in the application. The user reviews the 3D model and, if any modifications are necessary, enters specific instructions (e.g., "make the living room bigger"). Based on the modification instructions sent back to the server, the server regenerates a new 3D model and sends it back to the user's device. This process is repeated until the user is satisfied.

[1671] Virtual Tour

[1672] The user's device can display the generated 3D model in a VR space. The user can attach a smartphone to a VR headset and take a virtual tour. This allows the user to experience the interior of the designed home in real time and check the details.

[1673] Final order confirmation and construction management

[1674] Once the user is satisfied with the final model and places an order, the order data is sent to the server. The server aggregates the order information and sends it to the partner construction company. This process starts the construction project. The server periodically receives progress information from the construction company and notifies the user's device in real time. The user can check the project's progress (e.g., "foundation work completed") at any time through the application.

[1675] Specific examples

[1676] A user uses a smartphone app to input desired housing information. For example, they enter information such as "Style: Natural Modern," "Layout: 4LDK," "Budget: 50 million yen," and "Materials used: Wood, glass, concrete." The server receives this information and generates a 3D model using a generative AI model. The prompt statement "Please generate a 3D model of a 4LDK natural modern home based on the following information: Style: Natural Modern, Layout: 4LDK, Budget: 50 million yen, Materials used: Wood, glass, concrete" is passed to the generative AI model.

[1677] The user can view the generated 3D model in VR space, and if they feel there is a problem with the living space, they can input a correction instruction such as "make the living space larger." The server receives this correction instruction and updates the 3D model again using the generated AI model. This process is repeated until a model that satisfies the user is finally completed. Once the user confirms the final order, the construction project begins, and progress information is notified to the user in real time.

[1678] This system allows users to efficiently design their desired home and check and modify it in real time through virtual tours.

[1679] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1680] Step 1:

[1681] The user launches the smartphone app and inputs the desired housing characteristics (e.g., style, layout, budget, materials used, etc.). The input data is initially stored on the user's device and later sent to the server.

[1682] Input: House characteristic data entered by the user.

[1683] Output: House feature data temporarily stored on the user's device.

[1684] Step 2:

[1685] The device sends the user's input data to the server, which then analyzes the data. This analysis includes data validation and missing value completion.

[1686] Input: House characteristic data sent from the user terminal.

[1687] Output: Analyzed housing data stored on the server.

[1688] Step 3:

[1689] The server generates an initial 3D model using a generative AI model based on the analyzed data. A prompt is generated and input into the generative AI to generate the 3D model.

[1690] Input: Parsed housing data, prompt for the generative AI model.

[1691] Output: The generated 3D model data.

[1692] Step 4:

[1693] The server sends the generated 3D model data to the user's device, where the 3D model is displayed and the user can confirm the model.

[1694] Input: Generated 3D model data.

[1695] Output: The 3D model displayed on the user's device.

[1696] Step 5:

[1697] The user checks the 3D model and inputs correction instructions as necessary. The correction instructions are entered into the terminal as specific requests (e.g., "Make the living room bigger") and sent to the server.

[1698] Input: Correction instructions entered by the user.

[1699] Output: Correction instruction data sent to the server.

[1700] Step 6:

[1701] The server regenerates the 3D model based on the received correction instructions, and then passes the prompt sentence to the generative AI model again to generate an updated 3D model.

[1702] Input: User correction instruction data, update prompt sentence to the generative AI model.

[1703] Output: Updated 3D model data.

[1704] Step 7:

[1705] The server then sends the updated 3D model back to the user's device, where the user reviews the new 3D model and repeats this process until the modifications are complete.

[1706] Input: Updated 3D model data.

[1707] Output: The updated 3D model displayed on the user's device.

[1708] Step 8:

[1709] When the user finally completes a model that satisfies them, they press the order confirmation button, and the terminal sends the confirmed order data to the server.

[1710] Input: Final confirmed 3D model, order confirmation information.

[1711] Output: Order confirmation data sent to the server.

[1712] Step 9:

[1713] The server sends the order information to the construction company based on the received order confirmation data. It also starts managing the construction project, periodically receives progress information from the construction company, and notifies the user terminal.

[1714] Input: Order confirmation data, project progress information.

[1715] Output: Order information sent to the construction company, progress information notified to the user terminal.

[1716] Step 10:

[1717] The 3D model is displayed in a VR space so that users can take a virtual tour through their devices. Users can connect their smartphone to a VR headset and enjoy the virtual tour experience.

[1718] Input: 3D model data to be displayed in VR space.

[1719] Output: 3D model displayed in a VR headset for virtual tours.

[1720] Through these steps, users can design their desired home in detail, check, modify, and finalize the design in virtual space, and keep track of the project's progress in real time.

[1721] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1722] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[1723] User requirement collection and emotion recognition

[1724] A user downloads and installs the ordering app. When the user launches the app, a screen for entering basic information appears. The ordering app has the ability to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions during the process.

[1725] Generating 3D models and reflecting emotion data

[1726] The device sends the user's input data and the recognized emotion data to the server. The server then first analyzes the characteristic data of the user's desired home, and then analyzes the emotion data. Based on the analysis results, the generative AI then activates and generates an initial 3D model that takes the user's emotion data into account. This process makes it possible, for example, to design a home that incorporates elements that the user feels positive about.

[1727] Emotional reflection of 3D model checking and correction

[1728] The generated 3D model is sent to the device and displayed to the user. When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs corrections they would like to make to the model, their emotions are also collected at the same time. For example, along with instructions such as "I would like the living space to be a little larger," the user's emotions at that time (dissatisfaction, expectations, etc.) are sent.

[1729] The device sends these correction instructions and emotion data to the server, which then uses the generative AI to correct the 3D model. The emotion data is used to predict the importance of the user's request and makes optimal corrections based on that.

[1730] Final order confirmation

[1731] When the user is satisfied with the final model, they press the order confirmation button. The device then sends the final 3D model data, order information, and emotion data to the server. The server then aggregates the order details and sends the order data to the partner construction company. This confirmation information is then sent to the device as a notification to the user.

[1732] Emotional reflection in project management and progress notifications

[1733] Once an order is confirmed, the server uses that information to manage the progress of the construction project. The construction company starts the project based on the order data and periodically reports progress information to the server. The server receives this progress information and notifies the device at the set timing. The emotion engine also operates when notifications are sent, capturing the user's emotions when providing feedback and sending them to the server.

[1734] Specific examples

[1735] Example 1:

[1736] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[1737] Example 2:

[1738] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[1739] In this way, by incorporating an emotion engine, the system of the present invention can design a house that reflects the user's feelings, thereby achieving higher customer satisfaction.

[1740] The processing flow will be explained below.

[1741] Step 1:

[1742] A user downloads and installs the ordering app. When the user launches the app, a screen appears where the user can enter their information (name, contact details).

[1743] Step 2:

[1744] In addition to a screen for entering basic information, the device also activates an emotion engine that captures the user's facial expressions and voice. As the user enters information, the emotion engine captures the user's emotional data in real time.

[1745] Step 3:

[1746] The user inputs the home's characteristics (style, layout, budget, materials used, etc.) into the ordering app. The emotion engine captures the user's emotions while they are typing.

[1747] Step 4:

[1748] The device temporarily stores the user's input data and emotion data in a local database and then transmits it to the server.

[1749] Step 5:

[1750] The server receives and analyzes the user's input data, then analyzes the emotion data. Based on these analysis results, a generative AI is activated to generate an initial 3D model that takes the user's emotions into account.

[1751] Step 6:

[1752] The server sends the generated 3D model and the emotion analysis results to the device, which then displays the 3D model to the user and asks for their confirmation.

[1753] Step 7:

[1754] The user checks the 3D model and inputs the parts they want to modify. At the same time, the emotion engine captures the user's emotions and sends them, along with the modification instructions, from the device to the server.

[1755] Step 8:

[1756] The server receives the modification instructions and emotion data, and again modifies the 3D model using the generative AI. At this time, the model is adjusted with an emphasis on the user's emotion data. For example, it focuses on improving areas that the user expressed dissatisfaction with.

[1757] Step 9:

[1758] The server resends the modified 3D model to the device, which displays it to the user and asks for confirmation again. This process is repeated until the user is satisfied.

[1759] Step 10:

[1760] If the user is satisfied with the final model, they press the "Confirm Order" button, and the device sends the final 3D model data, order information, and emotion data to the server.

[1761] Step 11:

[1762] The server collects the order details and sends the order data to the affiliated construction company. This confirmation information is then sent to the terminal and notified to the user.

[1763] Step 12:

[1764] Construction companies start projects based on order data, and the server manages the progress of construction projects.

[1765] Step 13:

[1766] The server periodically sends progress information to the device. The device notifies the user of the progress and displays it. The emotion engine also works when notifying the user, capturing the emotions expressed by the user regarding the progress.

[1767] Step 14:

[1768] Users can check the progress and enter feedback as needed. For example, requests such as "I want to change the color of the exterior walls" are captured along with their emotions at the time.

[1769] Step 15:

[1770] The device sends the user's feedback and emotional data to a server, which analyzes the feedback and forwards it to the construction company.

[1771] Step 16:

[1772] The construction company makes corrections based on the feedback and performs a final check. The server sends a completion notification to the terminal and notifies the user of the date and time of handover. The user makes a final check and the handover is completed in a satisfactory manner.

[1773] Example 2

[1774] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1775] Conventional home design systems can collect specific user requirements, but do not consider the emotions felt by the user during the design process. As a result, the user's emotions and satisfaction are not reflected in the design process, which can lead to dissatisfaction with the final home design. In particular, there is a need for a system that can accurately reflect the user's emotions when making changes to the layout or style of the home.

[1776] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1777] In this invention, the server includes: a means for a user to input the characteristics of a home desired by the user; a means for the terminal to capture the user's facial expressions and voice in real time and recognize emotions; a means for the server to receive and analyze the user's input data and emotional data; a means for the server to generate a 3D model using a generation AI based on the analyzed data; a means for the server to send the generated 3D model data to the terminal; a means for the user to check the 3D model and input correction instructions; a means for the terminal to recapture the user's facial expressions and voice and recognize emotions; a means for the server to receive the correction instructions and emotional data and regenerate the 3D model; a means for the user to make a final confirmation and confirm the order; a means for the server to send the order data to a construction company and manage the project; and a means for the server to notify the terminal of project progress information. This makes it possible to reflect the user's emotions in the home design process and provide a final home design that will provide a high level of satisfaction.

[1778] "User" refers to an individual or corporation that uses the system to design a desired home.

[1779] "Terminal" refers to an electronic device used by a user, such as a computer, smartphone, or tablet.

[1780] "Facial and audio capture" refers to the process by which the device records the user's facial movements and voice in real time.

[1781] "Emotion recognition" refers to analyzing a user's emotional state (e.g., joy, sadness, anticipation, dissatisfaction) from captured facial and voice data.

[1782] "Server" refers to a central computing device that processes and manages data for the entire system.

[1783] "Input data" refers to information including the characteristics and requirements of the home desired by the user.

[1784] "Emotion data" refers to information about emotions recognized from the user's facial expressions and voice.

[1785] "Analyzing" refers to the server processing the received data with an analysis algorithm.

[1786] "Generative AI" refers to a system that uses artificial intelligence technology to generate a 3D model of a house based on user requirements and emotional data.

[1787] "3D model" refers to three-dimensional design data for a house that is generated based on the user's wishes and emotional data.

[1788] "Sending" refers to the process by which the server transfers the generated data to the terminal.

[1789] "Modification instructions" refer to requests for changes or adjustments made by the user to the 3D model.

[1790] "Regenerate" refers to the server recreating the 3D model based on the correction instructions and emotion data.

[1791] "Final confirmation" refers to when the user is satisfied with the final 3D model and confirms the order.

[1792] "Order Data" means all order information included with the final confirmed 3D model.

[1793] "Construction company" refers to the corporation that actually constructs the housing ordered.

[1794] "Managing the project" refers to the server monitoring and managing the progress of the entire home building process.

[1795] "Progress Information" means data relating to the progress of a construction project.

[1796] "Notifying" refers to the process by which the server communicates progress and other important information to the device.

[1797] This invention is a system that provides a more satisfying home by recognizing the user's emotions and reflecting that emotion data in the home design process. In addition to the process where the user inputs the characteristics of the home, this system incorporates an emotion engine to acquire the user's emotion data and generate an optimal 3D model using generative AI.

[1798] User requirement collection and emotion recognition

[1799] A user downloads and installs a home design app on their smartphone or tablet. When the app is launched, a screen for entering basic information appears. The app is equipped with a function to capture the user's facial expressions and voice in real time. As the user enters details such as their name, contact information, desired home style and layout, budget, and materials used, the emotion engine recognizes the user's emotions in real time. Specifically, the app uses the smartphone's camera and microphone to acquire emotion data using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[1800] Generating 3D models and reflecting emotion data

[1801] The device sends the user's input data and emotion data to the server. Upon receiving this data, the server first analyzes the user's request data, and then analyzes the emotion data. Based on the analysis results, the server activates the generative AI to generate an initial 3D model that reflects the user's emotion data. The server then sends the generated 3D model data to the device, and uses an interactive viewer (e.g., Three.js) to display the 3D model on the smartphone screen.

[1802] Emotional reflection of 3D model checking and correction

[1803] When the user checks the 3D model, the emotion engine is activated again, capturing the user's facial expressions and voice. When the user inputs the parts of the model they want to modify, their emotions are also collected at the same time. Specifically, the camera and microphone are used again to collect the user's emotional data. The device sends these modification instructions and emotional data to the server, and the server again uses the generative AI to modify the 3D model. The emotional data is used to determine the importance of the user's request, and the optimal modifications are made based on that.

[1804] Final order confirmation and project management

[1805] When the user is satisfied with the final model, they press the order confirmation button. The device sends the final 3D model data, order information, and emotion data to the server. The server consolidates the order details and sends the order data to the partner construction company. This confirmation information is also sent to the device as a notification to the user. Once the project begins, the server manages the progress of the construction project. The construction company periodically reports progress information to the server, and the server receives this progress information and notifies the device at the set time. The emotion engine also runs when notifying, capturing the emotion of the user's feedback and sending it to the server.

[1806] Examples and thought-provoking prompts

[1807] Example 1

[1808] A user requests a natural modern-style home with a 4LDK floor plan. When entering the necessary information through the app, the emotion engine captures the user's facial expressions and voice, and emotional data is also sent. Based on this, the server uses generative AI to provide the user with an initial 3D model. If the user expresses dissatisfaction with the size of the living space, this emotional data is reflected in the revision process. Finally, the user obtains a model they are satisfied with, confirms the order, and the project begins.

[1809] Prompt Sentence Examples

[1810] If a user looking for a natural modern 4LDK home expresses dissatisfaction with the size of the living space, please suggest how to improve it based on emotional data.

[1811] Example 2

[1812] Another user requests a minimalist 2LDK layout. The emotion engine captures emotions as the user sets the budget and materials to be used in detail through the app. The server uses generative AI based on this information to create a 3D model and sends it to the device. When the user reviews the model and requests a change to the bedroom position, this emotion is included. The server regenerates a revised model that reflects the emotion data, and the order is finally confirmed to the user's satisfaction. The user is then periodically notified of progress updates on the construction project.

[1813] Prompt Sentence Examples

[1814] If a user with a budget constraint requests to change the location of the bedroom in a minimalist 2LDK apartment, please suggest the optimal layout based on the sentiment data.

[1815] In this way, by incorporating an emotion engine, the system of the present invention can design a home that reflects the user's feelings, achieving higher customer satisfaction. The hardware used to implement this system includes a smartphone and a server, and specific software includes facial recognition software (e.g., OpenFace), voice analysis software (e.g., IBM Watson), and an interactive viewer (e.g., Three.js).

[1816] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1817] Step 1:

[1818] A user downloads and installs a home design app.

[1819] What it does: Download and install the app from the app store on your smartphone or tablet.

[1820] Input: None

[1821] Output: Device with home design app installed

[1822] Step 2:

[1823] The user launches the app and is presented with a screen for entering basic information.

[1824] Specific operation: When the user taps the app icon to launch the app, a screen for entering basic information appears on the device.

[1825] Input: None

[1826] Output: Basic information input screen

[1827] Step 3:

[1828] Users enter details such as their name, contact information, desired home style and layout, budget, and materials used.

[1829] Specific behavior: The user enters the required information into the form and proceeds to the next screen.

[1830] Input: User-entered house characteristics data

[1831] Output: The entered house feature data is saved on the device.

[1832] Step 4:

[1833] The device captures the user's facial expressions and voice in real time and recognizes their emotions.

[1834] Specific operation: The smartphone's camera and microphone are activated, and emotion data is obtained using facial recognition software (e.g., OpenFace) and voice analysis software (e.g., IBM Watson).

[1835] Input: User's facial expression and voice data

[1836] Output: Emotion data

[1837] Step 5:

[1838] The terminal transmits the user's input data and emotion data to the server.

[1839] Specific behavior: The input data and captured emotion data are sent to the server using the HTTPS protocol.

[1840] Input: House feature data, emotion data

[1841] Output: Data sent to the server

[1842] Step 6:

[1843] The server analyzes the received data and generates an initial 3D model using a generative AI model.

[1844] Specific operation: The server analyzes the received data and obtains the analysis results using a deep learning model (e.g., TensorFlow). Based on that data, it uses generative AI (e.g., Unity3D or Blender) to render a 3D model.

[1845] Input: House feature data, emotion data

[1846] Output: Initial 3D model

[1847] Step 7:

[1848] The server sends the generated 3D model data to the terminal.

[1849] Specific operation: The generated 3D model data is encoded and sent to the terminal using the HTTPS protocol.

[1850] Input: Initial 3D model

[1851] Output: 3D model sent to device

[1852] Step 8:

[1853] The device displays the 3D model to the user.

[1854] Specific operation: Use an interactive viewer (e.g., Three.js) to display a 3D model on the smartphone screen and allow the user to check it.

[1855] Input: 3D model data

[1856] Output: A displayed 3D model

[1857] Step 9:

[1858] The user checks the 3D model and inputs correction instructions.

[1859] Specific operation: The user adjusts the model using the touch interface of the smartphone.

[1860] Input: User correction instructions

[1861] Output: Correction instruction data

[1862] Step 10:

[1863] The device recaptures the user's facial expressions and voice to recognize emotions.

[1864] Specific operation: Again, the camera and microphone are used to collect user emotion data.

[1865] Input: User's facial expression and voice data

[1866] Output: Emotion data at the time of correction

[1867] Step 11:

[1868] The device sends correction instructions and emotion data to the server.

[1869] Specific operation: Input data and emotion data are sent to the server using the HTTPS protocol.

[1870] Input: Correction instruction data, emotion data when correcting

[1871] Output: Correction data and emotion data sent to the server

[1872] Step 12:

[1873] The server regenerates the 3D model based on the correction instructions and emotion data.

[1874] Specific operation: The server integrates the correction instructions and emotion data, and again uses the generative AI to generate a corrected 3D model.

[1875] Input: Correction instruction data, emotion data when correcting

[1876] Output: Modified 3D model

[1877] Step 13:

[1878] If the user is satisfied with the final model, he or she presses the order confirmation button.

[1879] Specific operation: The user presses the order confirmation button on the smartphone.

[1880] Input: User order confirmation operation

[1881] Output: Order confirmation data

[1882] Step 14:

[1883] The terminal sends the final 3D model data, order information, and emotion data to the server.

[1884] Specific behavior: The final data is sent to the server using the HTTPS protocol.

[1885] Input: Final 3D model data, order information, emotion data

[1886] Output: Final data sent to the server

[1887] Step 15:

[1888] The server collects the order information and sends the order data to the affiliated construction company.

[1889] Specific operation: The server sends the order data to the construction company via email or API.

[1890] Input: Order details data

[1891] Output: Order data sent to construction company

[1892] Step 16:

[1893] The server notifies the user of the order confirmation information.

[1894] Specific operation: The server sends a push notification to the device to notify the user that the order has been confirmed.

[1895] Input: Order confirmation information

[1896] Output: Push notification to the user

[1897] Step 17:

[1898] The server manages the progress of the construction project.

[1899] What it does: The server uses project management software (e.g. Jira) to track progress.

[1900] Input: Project Management Information

[1901] Output: Managed progress data

[1902] Step 18:

[1903] The construction company reports progress information to the server.

[1904] What happens: A construction company uses a progress reporting tool (e.g., Microsoft Project) to send progress data to a server.

[1905] Input: Progress information

[1906] Output: Progress data sent to the server

[1907] Step 19:

[1908] The server notifies the user of progress information.

[1909] Specific operation: The server sends a push notification to the device at the set timing.

[1910] Input: Progress information

[1911] Output: Progress notification to user

[1912] Step 20:

[1913] The device will display a notification and the emotion engine will capture the emotion during the feedback.

[1914] Specific behavior: When the user confirms the notification, the device again uses the camera and microphone to capture emotion data.

[1915] Input: Emotional data as user feedback

[1916] Output: Feedback emotion data sent to the server

[1917] (Application example 2)

[1918] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1919] The current home design process rarely takes into account user emotions, making it difficult to increase true user satisfaction. Virtual stores also lack personalized responses based on user emotions, creating a need for improved user experience. Conventional technologies lack the means to analyze user emotions in real time and customize products and services based on those emotions, making user satisfaction a challenge.

[1920] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input the characteristics of the house desired by the user; means for the terminal to capture the user's facial expressions and voice and acquire emotional data; means for the server to receive and analyze the user's input data and emotional data; means for the server to generate a 3D model based on the analyzed data and taking the emotional data into consideration using a generation AI; means for the server to send the generated 3D model data to the user terminal; means for the user to check the 3D model and input correction instructions; means for the terminal to re-capture the user's facial expressions and voice and acquire emotional data; means for the server to receive the correction instructions and emotional data and regenerate the 3D model; means for the user to make a final confirmation and confirm the order; means for the server to send the order data to the construction company and manage the project; and means for the server to notify the user terminal of project progress information. This makes it possible to provide more personalized products and services that reflect the user's emotions in real time.

[1921] The "means for the user to input the characteristics of the home desired" is a device or system that provides an interface for the user to specifically input the requirements and desires regarding the ideal home.

[1922] "Means for the device to capture the user's facial expressions and voice and obtain emotional data" refers to technology that captures the user's facial expressions and voice in real time, analyzes...

Claims

1. a means for a user to input desired home characteristics; means for the server to receive and analyze user input data; A means for the server to generate a 3D model using generation AI based on the analysis data; A means for the server to transmit the generated 3D model data to a user terminal; A means for the user to check the 3D model and input correction instructions; a means for the server to receive the correction instructions and regenerate the 3D model; A means for the user to finalize and confirm the order; a means for the server to transmit order data to construction companies and manage projects; A system including a means for a server to notify a user terminal of project progress information.

2. 2. The system of claim 1, further comprising means for a user to send feedback to the server through the terminal, and for the server to forward the feedback to the construction company.

3. 2. The system according to claim 1, further comprising means for the server to periodically notify the user terminal of progress information, and for the user to check the progress status.

Images