System

Abstract

A system is provided.SOLUTION: A system comprising: means for taking and uploading a picture of his / her room; means for analyzing characteristics of the room from the uploaded picture; means for displaying a generated interior design proposal; means for a user to adjust the displayed design proposal; means for calculating and settling a budget based on selected furniture and design; and means for delivering and assembling the selected furniture.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When designing a home's interior, many people have to consider various factors, such as room size, space utilization efficiency, multifunctionality, harmony with existing furniture, and budget management, leaving them with a lot of stress. It's also difficult to meet individual needs, such as updating the interior to match the season or trends, or taking into consideration pets and children. Furthermore, there is a lack of services that provide consistent support, from interior design ideas to furniture purchase and installation, making the overall process time-consuming. There is a need for a system that can solve these issues and allow for easy and efficient home interior design. [Means for solving the problem]

[0005] To solve the above problems, we provide a system that includes a means for taking and uploading photos of one's own room, a means for analyzing the characteristics of the room from the uploaded photos, a means for displaying generated interior design proposals, a means for the user to adjust the displayed design proposals, a means for calculating a budget and making payments based on the selected furniture and design, and a means for delivering and assembling the selected furniture. Furthermore, by including a means for automatically generating interior design proposals using a generative AI model based on the analyzed room characteristics, a means for customizing the interior design proposals based on the user's past preference data and current interior trends, and a means for automatically arranging and notifying the user of the delivery date and time of the selected furniture, we enable efficient and easy interior design.

[0006] "Photos" are image data taken with a camera to visually capture the characteristics and current state of a room.

[0007] "Upload" is the process of sending data from a terminal to a server.

[0008] "Analysis" is the process of automatically extracting and understanding room characteristics (dimensions, furniture arrangement, color, etc.) from given photo data.

[0009] A "generative AI model" is an algorithm that uses artificial intelligence technology to create new interior designs.

[0010] An "interior design proposal" is a furniture arrangement and decorative configuration suggested by the generative AI model to suit the user's room.

[0011] "Display" refers to the act of visually presenting information or design ideas to the user.

[0012] "Adjustment" is the process by which a user modifies the proposed interior design to suit their own preferences.

[0013] "Budget" is the financial information used to calculate the total cost of the furniture and design selections.

[0014] "Payment" is the process of making payment for products or services selected by a user.

[0015] "Delivery" refers to the act of transporting the purchased furniture to a location designated by the user.

[0016] "Assembly" is the process of setting up delivered furniture in the user's room and making it ready for use.

[0017] "Characteristics" refers to the physical and visual characteristics of a room.

[0018] "Customization" is the act of adjusting design and functionality to suit the individual needs and preferences of a user.

[0019] "Notifications" are messages that inform users of important information or updates. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention relates to a system that allows users to efficiently and easily design their home interiors. By taking and uploading photos of their own rooms, the system provides comprehensive support, from interior design proposals to furniture purchases and installation. The program for this system is specifically implemented as follows through interactions between a server, a terminal, and a user.

[0042] Detailed explanation of functions

[0043] 1. Take and upload a photo

[0044] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[0045] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[0046] 2. Interior design proposals

[0047] The server receives the uploaded photo data, and the image analysis module automatically analyzes characteristics such as room dimensions, furniture placement, and color hue.

[0048] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0049] 3. Adjust the design

[0050] The device displays the generated interior design proposals to the user in an interactive UI.

[0051] Users can view the displayed design proposals and adjust the color, placement, and type of furniture, and the system will reflect the user's changes in real time.

[0052] 4. Budget confirmation and settlement

[0053] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[0054] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent to the server and payment processing is carried out.

[0055] 5. Delivery and assembly services

[0056] The server checks the stock status of the purchased furniture and automatically adjusts the delivery schedule.

[0057] The server arranges delivery and assembly services and notifies the terminal of the delivery date and time.

[0058] The user receives the furniture on the scheduled delivery date, and the delivery staff assembles and installs the furniture.

[0059] Specific examples

[0060] The user takes three photos of their living room and follows the app's instructions to upload them.

[0061] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[0062] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[0063] The terminal displays the furniture prices and total budget to the user, who then makes the payment online.

[0064] The server arranges the delivery and assembly service schedule for the purchased furniture and notifies the terminal.

[0065] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[0066] As described above, this system allows users to propose and realize highly convenient interior designs through a series of processes.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[0070] Step 2:

[0071] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[0072] Step 3:

[0073] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0074] Step 4:

[0075] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[0076] Step 5:

[0077] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[0078] Step 6:

[0079] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[0080] Step 7:

[0081] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[0082] Step 8:

[0083] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0084] Step 9:

[0085] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0086] Step 10:

[0087] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[0088] Step 11:

[0089] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] Conventional interior design systems require users to manually input room dimensions and furniture layouts, making operation cumbersome and inefficient. Furthermore, if the proposed interior design does not match the user's preferences, readjusting or resubmitting the design is cumbersome, requiring time and effort.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes: a means for users to take and upload photos of their rooms; a means for the server to analyze the characteristics of the room from the uploaded photos; a means for the server to automatically generate interior design proposals using a generative AI model based on the analyzed characteristics; a means for the terminal to display the generated interior design proposals; a means for the user to adjust the displayed design proposals; a means for the terminal to calculate a budget based on the selected furniture and design and make payments; and a means for the server to deliver and assemble the selected furniture. This allows users to easily receive interior design proposals simply by uploading photos of their rooms, and the generative AI model automatically generates optimal design proposals taking into account the user's past preference data and current interior trends, allowing for adjustments in real time. This significantly reduces the amount of work required and enables efficient and satisfying interior design proposals.

[0095] "User" refers to an individual who uses this system to propose and realize interior designs.

[0096] "Server" refers to the computer system that receives the photo data, analyzes the images, generates interior design suggestions using generative AI models, and arranges for the delivery and assembly of the furniture.

[0097] "Device" refers to the device (e.g., smartphone or tablet) that a user uses to take and upload photos and view and adjust the generated interior design proposals.

[0098] "Photo data" refers to an image file that a user takes of their room and sends from their terminal to the server.

[0099] "Image analysis module" refers to a software module for automatically analyzing room characteristics from uploaded photo data.

[0100] "Generative AI model" refers to an artificial intelligence model used to generate interior design ideas based on analyzed room characteristics.

[0101] "Interior design proposal" refers to a series of design suggestions, such as furniture arrangements and color hues, generated by a generative AI model.

[0102] "User Interface" refers to the interactive screen on the terminal that allows a user to view and adjust interior design proposals.

[0103] "Budget calculation means" refers to a function for calculating the overall budget based on the furniture and design selected by the user.

[0104] "Payment method" refers to the function that allows users to pay for furniture and other items online.

[0105] "Delivery and assembly arrangement means" refers to a function for checking the inventory of furniture purchased by the user and arranging delivery and assembly.

[0106] "User's past preference data" refers to historical data about interior designs and furniture that the user has selected in the past.

[0107] "Interior trends" refers to interior design trends and styles that are currently popular in the market.

[0108] This invention relates to a system that allows users to efficiently and easily design their home interiors. Users can take and upload photos of their rooms, and the system provides comprehensive support from interior design proposals to furniture purchases and installation. This system is specifically implemented through interactions between a server, terminals, and users.

[0109] Hardware and Software

[0110] Server: This is the computer system that serves as the core of the system, and is generally a cloud server or dedicated server. It receives photo data, analyzes images, generates interior design proposals using generative AI models, and arranges for furniture delivery and assembly. For example, OpenCV is used for image analysis, and GPT-3 or DALL-E is used for generative AI models.

[0111] Terminal: A device operated by the user, typically a smartphone or tablet. Photos are taken using the device's camera and uploaded to a server via an application. The device also displays and adjusts interior design proposals, checks budgets, and processes payments.

[0112] User: An individual who uses the system to propose and realize interior designs. Users take photos of their rooms, select and arrange furniture based on the interior design proposals, and finally purchase the furniture.

[0113] Program processing

[0114] Taking and uploading photos

[0115] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. For example, they can take photos of the entire living room, the area around the sofa, and specific locations such as the position of the TV stand. The photos are temporarily saved on the device, and the user can press the upload button to send the selected photos to the server.

[0116] Interior design proposals

[0117] The server analyzes the received photo data using an image analysis module (e.g., OpenCV or TensorFlow). The analysis results include the room's dimensions, furniture layout, color hues, etc. Based on these analysis results, the server generates interior design proposals using a generative AI model (e.g., GPT-3 or DALL-E). This process also takes into account the user's past preference data and current interior trends.

[0118] Design Adjustments

[0119] The device displays the generated interior design proposals in an interactive user interface. The user can adjust the color, placement, and type of furniture based on the displayed design proposals. The system reflects the user's changes in real time, and the server instantly generates a new design proposal based on the changes and returns it to the device for display.

[0120] Budget confirmation and settlement

[0121] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The server receives the payment information and completes the payment process by connecting with an external payment provider.

[0122] Delivery and assembly services

[0123] The server checks the inventory status of the purchased furniture and automatically adjusts the delivery schedule. The confirmed delivery date and time is notified to the terminal, and the user picks up the furniture on the specified date. The delivery staff arrives and assembles and sets up the furniture according to the user's wishes.

[0124] Specific examples

[0125] The user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates modern-style interior design proposals. The generated design proposals are displayed on the device, allowing the user to change the color of the sofa from red to blue or move the TV stand to the other side of the window. The device then displays the furniture prices and total budget to the user, who then makes the online payment. The server then schedules delivery and assembly services for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[0126] Prompt Sentence Examples

[0127] "Analyze the photo of the room below and suggest some modern interior design ideas. The user wants the sofa to be red and the TV stand to be placed opposite the window."

[0128] As described above, this system allows users to propose and realize interior designs that are efficient and convenient through a series of processes.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Step 1: Take and upload a photo

[0131] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. Specifically, they take photos of the entire living room, the area around the sofa, and the position of the TV stand. The input is the captured image data (JPEG format).

[0132] The device temporarily stores the captured photos in its internal memory, after which the user can select the photos they want to upload on the upload screen.

[0133] The terminal provides an upload button. When the user presses the upload button, the selected photo data is sent to the server. The output is the image data sent to the server.

[0134] Step 2: Receiving and analyzing photo data

[0135] The server receives the image data. The input is the image data sent from the terminal.

[0136] The server launches an image analysis module (such as OpenCV or TensorFlow) and analyzes the room's characteristics (dimensions, furniture layout, hue, etc.) from the photo data. Specifically, edge detection and hue distribution calculations are performed. Data processing involves extracting features from the image and identifying the room's dimensions and furniture layout.

[0137] The analysis results (room dimensions, furniture layout, color tone, etc.) are saved as output and used in the next process.

[0138] Step 3: Generate interior design ideas

[0139] The server generates interior design proposals using a generative AI model (such as GPT-3 or DALL-E) based on the analysis results. The input is the analysis results from the image analysis module.

[0140] The server inputs a prompt to the generative AI model, for example, "Please suggest a design for a modern living room. The user wants the sofa to be red, and the TV stand to be placed opposite the window."

[0141] The generative AI model outputs interior design ideas. The output is a generated interior design idea.

[0142] Step 4: Displaying interior design ideas

[0143] The terminal displays the generated interior design proposal on a user interface. The input is the interior design proposal sent from the server.

[0144] The device displays design proposals to the user through an interactive UI, allowing them to freely rotate and zoom in and out of the entire room using 3D rendering.

[0145] The output is an interior design proposal displayed to the user.

[0146] Step 5: Adjust the design

[0147] The user can adjust the color, placement, type, etc. of the furniture based on the displayed design proposal. Input is via an interactive UI on the device.

[0148] The terminal transmits the user's adjustments to the server in real time, and the output is the adjustment data transmitted to the server.

[0149] The server receives the adjustments and generates a new design proposal, which is then sent back to the device.

[0150] The output is a revised interior design proposal.

[0151] Step 6: Budget review and closing

[0152] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user. The input is the furniture information sent from the server.

[0153] The terminal calculates the total budget and presents it to the user. The budget calculation is based on the price information of the selected furniture. The output is the total budget presented to the user.

[0154] The user confirms the budget and moves to the online payment screen. They enter the necessary payment information (credit card number, etc.) and make the payment.

[0155] The server receives the payment information and completes the payment process with the external payment provider. The output is a confirmation of successful payment.

[0156] Step 7: Delivery and assembly service

[0157] The server checks the stock status of the purchased furniture. The stock information is stored in a database and is searched using an SQL query. The input is the user's order information.

[0158] After the server checks the inventory, it automatically adjusts the delivery schedule. It works with the Delivery Administrator API to set the optimal delivery date. The output is the confirmed delivery date and time.

[0159] The server arranges delivery and assembly services, notifies the terminal of the confirmed delivery date and time, and outputs the delivery information notified to the user.

[0160] The user receives the furniture on the scheduled delivery date. The delivery staff arrives at the specified time and assembles and installs the purchased furniture. The output is a confirmation that assembly and installation have been completed.

[0161] (Application example 1)

[0162] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] While there are currently support systems that offer a complete range of services from interior design proposals to furniture purchases and installation, there are few ways for users to visually check the actual furniture placement and adjust it in real time. This often leads to problems such as the furniture not matching the image after purchase, making it difficult for users to design their home interiors without stress.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0165] In this invention, the server includes means for acquiring and uploading images of the user's interior, means for analyzing the characteristics of the interior from the uploaded images, means for displaying the generated interior design, means for the user to adjust the displayed design, means for calculating costs and making payments based on the selected furniture and design, means for delivering and assembling the selected furniture, and means for visually displaying the generated design in augmented reality using a smart device. This allows the user to visually check the actual furniture placement and adjust it in real time, easily achieving the optimal interior design.

[0166] "Means for acquiring and uploading images of one's own room" refers to a function that allows a user to take a photo of the room using a smartphone or camera and send the image data to the system.

[0167] "Means for analyzing the interior features from uploaded images" refers to a function in which the system analyzes uploaded image data and extracts features such as the interior dimensions, furniture arrangement, and color hue.

[0168] The "means for displaying the generated interior design" is a function for visually presenting to the user an interior design proposal generated based on the analyzed data.

[0169] "Means for users to adjust the displayed design" refers to a function that provides an interface that allows users to change the furniture placement, color, type, etc. of the presented design proposal.

[0170] "Means for calculating costs and making payments based on the selected furniture and design" is a function that calculates the total cost based on the furniture and design selected by the user and allows payment to be made online.

[0171] "Means for delivering and assembling selected furniture" is a function that arranges for the furniture purchased by the user to be delivered to the specified address and properly assembled and installed.

[0172] "Means for visually displaying the generated design in augmented reality using a smart device" refers to a function that uses devices such as smart glasses or a headset to display the generated interior design superimposed on the actual room, allowing the user to check its layout in real time.

[0173] This invention is a system that allows users to take pictures of their own rooms, propose and adjust interior designs based on the images, and ultimately purchase and install furniture efficiently. Specific embodiments of this system are described below.

[0174] First, the user takes a photo of their room using a device such as a smartphone or tablet. The photo is temporarily saved on the device and then sent to the server by pressing the upload button. At this time, the image data is transferred in an appropriate format.

[0175] The server then receives the uploaded images and uses an image analysis module to automatically analyze characteristics such as the room's dimensions, furniture placement, and color. Based on the analysis results, a generative AI model automatically generates interior design proposals. The output of the generative AI model is adjusted taking into account the user's past preference data and current interior design trends.

[0176] The generated interior design proposals are displayed in real time on smart devices. For example, if the user is wearing smart glasses, the generated design is overlaid on the real room using augmented reality (AR) technology, allowing the user to visually confirm the furniture and design as if they were actually placed in the room.

[0177] If users like the design, they can adjust the color, placement, and type of furniture. These adjustments are made through an interactive UI, with real-time feedback.

[0178] The terminal also displays detailed information about the selected furniture (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The payment process is handled by the server.

[0179] Finally, the server checks the inventory status of the purchased furniture and automatically schedules the delivery. It also arranges for delivery and assembly services and notifies the user of the delivery date and time. The user picks up the furniture on the scheduled delivery date, and the delivery staff assembles and sets it up.

[0180] For example, a user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's smart glasses, and the user can review it and change the color of the sofa from red to blue and move the TV stand to the other side of the window. Finally, the user confirms the furniture price and total budget, and makes an online payment. The delivery staff delivers the furniture on the specified date and assembles it according to the user's wishes.

[0181] An example of input to a generative AI model is the following prompt:

[0182] "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m."

[0183] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0184] Step 1: The user takes a photo of their room using a smartphone or tablet.

[0185] Input: A photo of an interior taken with a smartphone or tablet.

[0186] Output: The captured photos are temporarily saved on the device.

[0187] Specific behavior: The user launches the camera app and takes several photos of the room. If necessary, the app will display guidance on how to take photos properly.

[0188] Step 2: The user uploads a photo to the server.

[0189] Input: The indoor photo taken in step 1.

[0190] Output: Photo data sent to the server by pressing the upload button.

[0191] How it works: The user clicks the upload button in the app, and the selected photos are sent to the server. The photo data is transferred in a standard format such as JPEG.

[0192] Step 3: The server analyzes the uploaded image.

[0193] Input: Photo data of the room sent to the server.

[0194] Output: Analyzed interior features such as dimensions, furniture layout, and color.

[0195] Specific operation: The server uses the image analysis module to analyze the dimensions, furniture arrangement, and color of the photo, and as a result, obtains the interior feature data.

[0196] Step 4: The server uses the generative AI model to generate interior design proposals.

[0197] Input: Analyzed indoor feature data, user's past preference data, current interior trends.

[0198] Output: Generated interior design proposals.

[0199] How it works: The server uses the generative AI model to generate interior design proposals based on a prompt such as "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m." The output design proposals are saved in a digital format.

[0200] Step 5: The device displays the generated interior design proposal to the user.

[0201] Input: Generated interior design proposal.

[0202] Output: Interior design proposal displayed on the user's device.

[0203] How it works: Smart glasses, tablets, and other devices receive the design proposals and use augmented reality (AR) technology to overlay them onto the user's field of vision, allowing the user to view the design proposals through their device.

[0204] Step 6: The user adjusts the design proposal.

[0205] Input: Adjustment instructions from the user (changes to furniture color, placement, type, etc.).

[0206] Output: Adjusted interior design proposal.

[0207] Specific operation: The user operates the device to change the color, arrangement, type, etc. of the furniture in the design proposal. The changes are reflected in real time, and the final design proposal is updated.

[0208] Step 7: The device will display detailed information about the selected furniture and calculate your budget.

[0209] Input: Adjusted interior design proposal.

[0210] Output: Furniture details (price, brand, dimensions, etc.), calculated budget.

[0211] Specific operation: The device displays detailed information about the selected furniture based on the adjusted design proposal. The total cost and individual costs are also calculated and presented to the user.

[0212] Step 8: The user makes an online payment.

[0213] Input: Final interior design proposal and adjusted budget.

[0214] Output: Payment completion notification.

[0215] What happens: The user presses the payment button on the device to complete the online payment. The payment is made through a secure protocol, and a notification is sent to the server upon success.

[0216] Step 9: The server arranges furniture delivery and assembly services.

[0217] Input: Payment completion notification, inventory information.

[0218] Output: Notification of delivery date and time and arrangement of assembly staff.

[0219] Specific operation: The server checks the furniture inventory and schedules the optimal delivery date and assembly service. The delivery date and time are notified to the user's device, and delivery and assembly work are carried out on the specified date.

[0220] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0221] This invention relates to a system that allows for efficient and easy home interior design, and in particular provides more user-friendly interior designs by combining an emotion engine that recognizes the user's emotions and reflects them in design proposals. The program for this system is specifically implemented as follows through interactions between a server, terminals, and users, and the use of the emotion engine.

[0222] Detailed explanation of functions

[0223] 1. Take and upload a photo

[0224] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[0225] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[0226] 2. Interior design proposals

[0227] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0228] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0229] 3. Adjust the design

[0230] The device displays the generated interior design proposals to the user in an interactive UI.

[0231] The user can view the displayed design proposal and adjust the color, placement, type, etc. of the furniture. The device reflects the user's changes in real time and displays the updated design.

[0232] 4. Utilizing the Emotion Engine

[0233] To recognize the user's emotions, the device uses an emotion engine, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[0234] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0235] 5. Budget confirmation and settlement

[0236] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[0237] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0238] 6. Delivery and assembly services

[0239] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0240] The user prepares to receive the furniture on the scheduled delivery date.

[0241] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[0242] Specific examples

[0243] The user takes three photos of their living room and follows the app's instructions to upload them.

[0244] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[0245] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[0246] The device reads the user's facial expression, and the emotion engine recognizes that the user is in a relaxed state, and sends this to the server.

[0247] Based on the data from the emotion engine, the server will suggest more subdued colors and relaxing furniture.

[0248] The user checks the furniture price and total budget and makes an online payment. The server schedules the delivery and assembly service of the purchased furniture and notifies the terminal.

[0249] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[0250] As described above, this system provides users with the proposal and realization of highly convenient interior designs through a series of processes including the emotion engine.

[0251] The processing flow will be explained below.

[0252] Step 1:

[0253] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[0254] Step 2:

[0255] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[0256] Step 3:

[0257] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0258] Step 4:

[0259] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[0260] Step 5:

[0261] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[0262] Step 6:

[0263] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[0264] Step 7:

[0265] The device reads the user's facial expressions and tone of voice, and the emotion engine recognizes their current emotional state. For example, if the user is smiling, the emotion is determined to be "joy."

[0266] Step 8:

[0267] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0268] Step 9:

[0269] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[0270] Step 10:

[0271] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0272] Step 11:

[0273] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0274] Step 12:

[0275] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[0276] Step 13:

[0277] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[0278] Example 2

[0279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] Conventional interior design systems have difficulty proposing personalized designs that reflect the user's emotions. Furthermore, selecting and arranging furniture by oneself can be time-consuming, which can lead to a lack of completeness in the design. Furthermore, the complicated procedures involved in calculating costs and arranging furniture delivery and assembly reduce user convenience.

[0281] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing the characteristics of the space from the transmitted image, a means for automatically generating layout proposals using a generative AI model, and a means for recognizing the user's emotional state and adjusting the proposals based on that. This enables personalized interior design that reflects the user's emotions, and also enables efficient and consistent design adjustments, cost calculations, and furniture delivery and assembly procedures.

[0282] "Image of space" is video information taken by a user to show the situation of his / her specific location.

[0283] "Means of transmission" refers to the technology that transfers images of the space taken by the user to a server using wireless communication or the Internet.

[0284] "Spatial characteristics" refers to information extracted through image analysis, such as spatial dimensions, furniture arrangement, color, and lighting.

[0285] The "means of analysis" refers to technology that automatically extracts spatial characteristics using image analysis modules and artificial intelligence algorithms.

[0286] The "layout proposal" is a plan that proposes the optimal furniture layout and interior design based on the analyzed characteristics of the space.

[0287] The "display means" refers to a technology for visually presenting the generated layout plan to the user, and includes, for example, a mobile app or a web app.

[0288] "Adjustment means" refers to techniques that allow users to interactively change and adapt the displayed layout plan.

[0289] "Cost" refers to the total cost of the selected furniture and interior.

[0290] The "processing means" refers to the technology that calculates the cost of the furniture and interior items selected by the user and allows for online payment.

[0291] "Delivery and assembly means" refers to the technology and services for transporting the selected furniture to the customer's designated location and assembling it on-site.

[0292] A "generative AI model" is an artificial intelligence algorithm with natural language processing and image generation capabilities, and is a model for automatically generating placement plans.

[0293] "Emotional state" refers to the mental and emotional state of the user ascertained by analyzing facial expressions and tone of voice.

[0294] The "means of recognition" is a technology that uses an emotion engine to identify the user's emotional state.

[0295] This invention is a system that proposes personalized interior design taking into account the user's emotions. This system is realized using the interaction of a server, a terminal, and a user. Below, we will explain how data is processed using specific hardware and software.

[0296] First, a user takes multiple photos of their room using a device such as a smartphone or tablet. These photos are temporarily stored on the device, and the user can select one. The user then presses the upload button on the device to send the selected photos to the server. The server then saves the received photo data in an appropriate format (e.g., JPEG or PNG).

[0297] The server then launches an image analysis module to analyze the received photo data. Specifically, image processing libraries such as OpenCV and TensorFlow can be used. The analysis module automatically extracts characteristics such as room dimensions, furniture placement, and color hues. This extracted data is used to create interior design proposals.

[0298] After the analysis is complete, the server uses a generative AI model to automatically generate interior design proposals. Examples of such models include DALL-E and GPT-4. The design generation takes into account not only the analyzed room characteristic data, but also the user's past preference data and current interior trends. An example of a prompt sentence input to the generative AI model is as follows:

[0299] Example prompt sentence:

[0300] Room size: 5m x 6m. Furniture arrangement: Sofa, TV stand, coffee table. Preferred style: Modern. Favorite colors: Blue, gray. Current interior trends: Natural materials, simple design.

[0301] The generated interior design proposal is displayed to the user through the device's interactive user interface (UI). The user can interactively adjust the color, placement, and type of furniture in the displayed design proposal. For example, the user can change the color of a sofa from red to blue, or move a TV stand to the other side of a window. The device then reflects the user's changes in real time and redisplays the updated design proposal.

[0302] The device then uses an emotion engine to analyze the user's facial expressions and tone of voice. The emotion engine uses technologies such as the Emotion API from Microsoft Azure Cognitive Services. The emotional data obtained as a result of the analysis contains information such as whether the user is relaxed or excited. Based on this, the server can further adjust the design proposal. For example, if the user is in a relaxed state, the system will suggest calming colors and relaxing furniture.

[0303] Finally, the terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user confirms the budget and, if there are no problems, makes an online payment. The payment information is sent from the terminal to the server, and the payment is processed. The server then arranges for delivery and assembly services, and notifies the terminal of the detailed delivery date and time.

[0304] The customer prepares to receive the furniture on the specified delivery date. The delivery staff delivers the furniture to the customer's address on the specified date and assembles it on site. The customer checks the completed furniture and evaluates whether the service was satisfactory.

[0305] Through the above process, the system provides users with an easy and efficient way to propose and realize interior designs. By utilizing the emotion engine, more personalized design proposals are provided, improving user satisfaction.

[0306] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0307] Step 1:

[0308] A user launches the camera app on their smartphone or tablet and takes a photo of their room. They take multiple photos of different rooms, such as the living room and bedroom. These photo data are temporarily stored on the device and the user can select one. The input is the captured photo data, and the output is the selected photo data.

[0309] Step 2:

[0310] The device displays a photo upload button, and the user presses the upload button to select a photo. The device converts the selected photo into an appropriate format (e.g., JPEG, PNG) and sends it to the server. The input is the selected photo data, and the output is the photo data sent to the server.

[0311] Step 3:

[0312] The server stores the received photo data in storage. Next, the server launches an image analysis module (e.g., OpenCV, TensorFlow) to analyze the photo data. The analysis module identifies the room's dimensions, furniture layout, color hue, etc. The input is the received photo data, and the output is the analyzed room characteristic data.

[0313] Step 4:

[0314] The server uses the analyzed room characteristic data to launch a generative AI model (e.g., DALL-E, GPT-4) and automatically generate interior design proposals. The generation process also takes into account the user's past preference data and current interior trends. The input is the analyzed room characteristic data and the user's preference data, and the output is the generated interior design proposals.

[0315] Step 5:

[0316] The device displays the generated interior design proposal to the user in an interactive user interface (UI), allowing the user to check the displayed design proposal. The input is the generated interior design proposal, and the output is the design proposal displayed to the user.

[0317] Step 6:

[0318] The user adjusts the design proposal using the UI. For example, the user may change the color of the sofa from red to blue or rearrange the furniture. The device reflects the user's changes in real time and redisplays the updated design proposal. The input is the user's adjustment data, and the output is the updated design proposal.

[0319] Step 7:

[0320] The device uses the built-in camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state. This is done using an emotion engine (e.g., Microsoft Azure's Emotion API). The input is the user's facial expression and voice data, and the output is the recognized emotional state data.

[0321] Step 8:

[0322] The server further adjusts the interior design proposal based on the emotional state data obtained from the emotion engine. For example, if the user is in a relaxed state, it will suggest calming colors and relaxing furniture. The input is the emotional state data, and the output is the adjusted interior design proposal.

[0323] Step 9:

[0324] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the selected furniture data, and the output is the calculated budget data.

[0325] Step 10:

[0326] The user checks the budget and, if there are no problems, presses the online payment button. The terminal sends the payment information to the server, which then processes the payment. The input is the payment information, and the output is payment completion data.

[0327] Step 11:

[0328] The server checks the inventory and sets the delivery date for the purchased furniture. The server arranges the delivery and assembly service and notifies the terminal of the details. The input is the purchased furniture data, and the output is the delivery schedule data.

[0329] Step 12:

[0330] The user receives the furniture on the specified delivery date. The delivery staff brings the furniture to the user's address and assembles it on site. The user checks the completed furniture and rates whether the service was satisfactory. The input is information about the delivery and assembly service, and the output is the user's satisfaction rating data.

[0331] (Application example 2)

[0332] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] Conventional interior design systems have the problem of reducing user satisfaction because they propose designs based solely on the physical characteristics of the room and past preference data, without taking the user's emotional state into consideration. In particular, the interior design of autonomous vehicles requires providing a design that is optimal for the user's emotional state while traveling, but this issue has not been adequately resolved.

[0334] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking and uploading a photo of the user's room, means for analyzing characteristics of the room from the uploaded photo, means for displaying generated interior design proposals, means for the user to adjust the generated interior design proposals, means for recognizing the user's emotional state, means for adjusting the interior design proposals based on the emotional data, means for calculating a budget based on the selected furniture and design and making payments, and means for delivering and assembling the selected furniture. This makes it possible to provide an interior design that is optimal for the user's emotional state in real time, thereby improving satisfaction.

[0335] "Means to take and upload photos of your room" is a function that allows you to take photos of your room or the inside of your car using a smartphone or camera and transfer the image data to the system's server.

[0336] "Means for analyzing room characteristics from uploaded photos" is a function for analyzing received photo data and extracting the dimensions, furniture arrangement, color tone, etc. of a room or vehicle interior.

[0337] The "means for displaying the generated interior design proposal" is a function for visually displaying the interior design proposal generated by the server on the user's terminal.

[0338] The "means for the user to adjust the generated interior design proposal" is a function that allows the user to change the color, layout, materials, etc. of the displayed interior design proposal.

[0339] The "means for recognizing the user's emotional state" is a function for analyzing the user's facial expression, tone of voice, etc., to identify the user's emotional state at that time.

[0340] The "means for adjusting interior design proposals based on emotional data" is a function for further optimizing interior design proposals based on the recognized emotional data.

[0341] "Means for calculating a budget and making payments based on the selected furniture and design" is a function that calculates the overall budget based on the furniture and design selected by the user and makes online payments within that budget.

[0342] The "means for delivering and assembling the selected furniture" is a function for delivering the determined furniture to the user and assembling the furniture.

[0343] "Means for automatically generating interior design proposals using a generative AI model based on the analyzed characteristics of a room" is a function that uses a generative AI model to automatically propose interior designs based on the analysis results of a room or car interior.

[0344] "Means for customizing interior design proposals based on the user's past preference data and current interior trends" is a function for customizing interior design proposals taking into account the user's past preference data and the latest interior trends.

[0345] This invention is a system for efficiently and easily customizing the interior design of an autonomous vehicle. In particular, it provides a more user-friendly interior design by combining an emotion engine that recognizes the user's emotional state and reflects it in the design proposal.

[0346] The system program is specifically implemented as follows through the interaction between the server, the terminal, and the user and the use of the emotion engine.

[0347] Taking and uploading photos

[0348] Users can take photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device and can be selected by the user.

[0349] The device provides an upload button to send the user-selected photos to the server, where the photo data is transferred in the appropriate format.

[0350] Interior design proposals

[0351] The server then launches an image analysis module to analyze the received photo data, automatically extracting characteristics such as room or vehicle interior dimensions, furniture placement, and color hues.

[0352] The server uses a generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0353] Design Adjustments

[0354] The device displays the generated interior design proposal to the user in an interactive UI. Using a smartphone or head-mounted display, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture. The device reflects the user's changes in real time and displays the updated design.

[0355] Utilizing the Emotion Engine

[0356] The device uses an emotion engine to recognize the user's emotions, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[0357] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0358] Budget confirmation and payment

[0359] The terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.) and calculates the total budget and presents it to the user.

[0360] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0361] Delivery and assembly services

[0362] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0363] The user prepares to receive the furniture on the scheduled delivery date.

[0364] The delivery staff will deliver the furniture to the user's address at the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[0365] Specific examples

[0366] The user takes three photos of the interior of the autonomous vehicle and uploads them as instructed by the app. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's device. The user changes the color of the sofa from red to blue and moves the TV stand to the opposite side of the window. The device reads the user's facial expression and the emotion engine recognizes that the user is in a relaxed state, and sends this information to the server. Based on the data from the emotion engine, the server suggests more muted colors and more relaxing furniture. The user checks the price and total budget of the furniture and makes an online payment. The server arranges a delivery and assembly service schedule for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[0367] An example of a prompt for the generative AI model is, "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state." In this way, the system can provide an interior design that best suits the user's emotional state in real time, improving satisfaction.

[0368] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0369] Step 1:

[0370] The user takes photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device. The input is a photo of each location, and the output is the photo data stored on the device. At this stage, the user can take multiple photos as needed and select the appropriate photo according to the application's instructions.

[0371] Step 2:

[0372] The device provides an upload button and sends the photos selected by the user to the server. The input here is the photo data selected by the user, and the output is the photo data sent to the server. When the user presses the upload button, the device converts the photo data into an appropriate format and transfers it to the server.

[0373] Step 3:

[0374] The server launches an image analysis module to analyze the received photo data. The input is the photo data stored on the server, and the output is characteristics such as the dimensions of the room or car interior, furniture arrangement, and color hue. The server uses image analysis algorithms to extract these characteristics from the photo data and prepare it for further processing.

[0375] Step 4:

[0376] The server uses a generative AI model to generate interior design proposals based on the analysis results. The input is the analyzed characteristic data of the room or car interior, and the output is an interior design proposal. In particular, the user's past preference data and current interior trends are also taken into consideration. For example, a prompt sentence such as "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state" is input into the generative AI model.

[0377] Step 5:

[0378] The device displays the generated interior design proposal to the user in an interactive UI. The input is the interior design proposal sent from the server, and the output is the design proposal displayed on the device. At this stage, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture.

[0379] Step 6:

[0380] The device uses an emotion engine to recognize the user's emotions. Here, the device analyzes the user's facial expressions and tone of voice to identify their current emotional state. The input is the user's audio and video data, and the output is the analyzed emotional state data.

[0381] Step 7:

[0382] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. The input is the user's emotional state data and the interior design proposal, and the output is the adjusted design proposal. For example, if the user feels like relaxing, the server will suggest calming colors and relaxing furniture.

[0383] Step 8:

[0384] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the adjusted interior design plan and detailed furniture information, and the output is the calculated budget and displayed information.

[0385] Step 9:

[0386] The user checks the budget and makes an online payment. The input is the calculated budget and the user's payment information, and the output is payment confirmation data. The payment details are sent from the terminal to the server, which then processes the payment.

[0387] Step 10:

[0388] The server checks the inventory status of the purchased furniture and sets a delivery date. The server also arranges for delivery and assembly services and notifies the terminal of the delivery date and time. The input is furniture inventory data and the user's desired delivery date and time, and the output is delivery and assembly schedule information.

[0389] Step 11:

[0390] The user prepares to receive the furniture on the scheduled delivery date. The delivery staff delivers the furniture to the user's address on the specified date and time and assembles it. The input is the schedule information and the furniture itself, and the output is a report that the assembly is complete. The user checks the assembled furniture and evaluates whether the service was satisfactory.

[0391] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0392] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0393] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0394] [Second embodiment]

[0395] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0396] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0397] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0398] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0399] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0400] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0401] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0402] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0403] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0404] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0405] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0406] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0407] This invention relates to a system that allows users to efficiently and easily design their home interiors. By taking and uploading photos of their own rooms, the system provides comprehensive support, from interior design proposals to furniture purchases and installation. The program for this system is specifically implemented as follows through interactions between a server, a terminal, and a user.

[0408] Detailed explanation of functions

[0409] 1. Take and upload a photo

[0410] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[0411] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[0412] 2. Interior design proposals

[0413] The server receives the uploaded photo data, and the image analysis module automatically analyzes characteristics such as room dimensions, furniture placement, and color hue.

[0414] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0415] 3. Adjust the design

[0416] The device displays the generated interior design proposals to the user in an interactive UI.

[0417] Users can view the displayed design proposals and adjust the color, placement, and type of furniture, and the system will reflect the user's changes in real time.

[0418] 4. Budget confirmation and settlement

[0419] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[0420] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent to the server and payment processing is carried out.

[0421] 5. Delivery and assembly services

[0422] The server checks the stock status of the purchased furniture and automatically adjusts the delivery schedule.

[0423] The server arranges delivery and assembly services and notifies the terminal of the delivery date and time.

[0424] The user receives the furniture on the scheduled delivery date, and the delivery staff assembles and installs the furniture.

[0425] Specific examples

[0426] The user takes three photos of their living room and follows the app's instructions to upload them.

[0427] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[0428] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[0429] The terminal displays the furniture prices and total budget to the user, who then makes the payment online.

[0430] The server arranges the delivery and assembly service schedule for the purchased furniture and notifies the terminal.

[0431] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[0432] As described above, this system allows users to propose and realize highly convenient interior designs through a series of processes.

[0433] The processing flow will be explained below.

[0434] Step 1:

[0435] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[0436] Step 2:

[0437] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[0438] Step 3:

[0439] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0440] Step 4:

[0441] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[0442] Step 5:

[0443] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[0444] Step 6:

[0445] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[0446] Step 7:

[0447] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[0448] Step 8:

[0449] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0450] Step 9:

[0451] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0452] Step 10:

[0453] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[0454] Step 11:

[0455] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[0456] Example 1

[0457] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0458] Conventional interior design systems require users to manually input room dimensions and furniture layouts, making operation cumbersome and inefficient. Furthermore, if the proposed interior design does not match the user's preferences, readjusting or resubmitting the design is cumbersome, requiring time and effort.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0460] In this invention, the server includes: a means for users to take and upload photos of their rooms; a means for the server to analyze the characteristics of the room from the uploaded photos; a means for the server to automatically generate interior design proposals using a generative AI model based on the analyzed characteristics; a means for the terminal to display the generated interior design proposals; a means for the user to adjust the displayed design proposals; a means for the terminal to calculate a budget based on the selected furniture and design and make payments; and a means for the server to deliver and assemble the selected furniture. This allows users to easily receive interior design proposals simply by uploading photos of their rooms, and the generative AI model automatically generates optimal design proposals taking into account the user's past preference data and current interior trends, allowing for adjustments in real time. This significantly reduces the amount of work required and enables efficient and satisfying interior design proposals.

[0461] "User" refers to an individual who uses this system to propose and realize interior designs.

[0462] "Server" refers to the computer system that receives the photo data, analyzes the images, generates interior design suggestions using generative AI models, and arranges for the delivery and assembly of the furniture.

[0463] "Device" refers to the device (e.g., smartphone or tablet) that a user uses to take and upload photos and view and adjust the generated interior design proposals.

[0464] "Photo data" refers to an image file that a user takes of their room and sends from their terminal to the server.

[0465] "Image analysis module" refers to a software module for automatically analyzing room characteristics from uploaded photo data.

[0466] "Generative AI model" refers to an artificial intelligence model used to generate interior design ideas based on analyzed room characteristics.

[0467] "Interior design proposal" refers to a series of design suggestions, such as furniture arrangements and color hues, generated by a generative AI model.

[0468] "User Interface" refers to the interactive screen on the terminal that allows a user to view and adjust interior design proposals.

[0469] "Budget calculation means" refers to a function for calculating the overall budget based on the furniture and design selected by the user.

[0470] "Payment method" refers to the function that allows users to pay for furniture and other items online.

[0471] "Delivery and assembly arrangement means" refers to a function for checking the inventory of furniture purchased by the user and arranging delivery and assembly.

[0472] "User's past preference data" refers to historical data about interior designs and furniture that the user has selected in the past.

[0473] "Interior trends" refers to interior design trends and styles that are currently popular in the market.

[0474] This invention relates to a system that allows users to efficiently and easily design their home interiors. Users can take and upload photos of their rooms, and the system provides comprehensive support from interior design proposals to furniture purchases and installation. This system is specifically implemented through interactions between a server, terminals, and users.

[0475] Hardware and Software

[0476] Server: This is the computer system that serves as the core of the system, and is generally a cloud server or dedicated server. It receives photo data, analyzes images, generates interior design proposals using generative AI models, and arranges for furniture delivery and assembly. For example, OpenCV is used for image analysis, and GPT-3 or DALL-E is used for generative AI models.

[0477] Terminal: A device operated by the user, typically a smartphone or tablet. Photos are taken using the device's camera and uploaded to a server via an application. The device also displays and adjusts interior design proposals, checks budgets, and processes payments.

[0478] User: An individual who uses the system to propose and realize interior designs. Users take photos of their rooms, select and arrange furniture based on the interior design proposals, and finally purchase the furniture.

[0479] Program processing

[0480] Taking and uploading photos

[0481] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. For example, they can take photos of the entire living room, the area around the sofa, and specific locations such as the position of the TV stand. The photos are temporarily saved on the device, and the user can press the upload button to send the selected photos to the server.

[0482] Interior design proposals

[0483] The server analyzes the received photo data using an image analysis module (e.g., OpenCV or TensorFlow). The analysis results include the room's dimensions, furniture layout, color hues, etc. Based on these analysis results, the server generates interior design proposals using a generative AI model (e.g., GPT-3 or DALL-E). This process also takes into account the user's past preference data and current interior trends.

[0484] Design Adjustments

[0485] The device displays the generated interior design proposals in an interactive user interface. The user can adjust the color, placement, and type of furniture based on the displayed design proposals. The system reflects the user's changes in real time, and the server instantly generates a new design proposal based on the changes and returns it to the device for display.

[0486] Budget confirmation and settlement

[0487] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The server receives the payment information and completes the payment process by connecting with an external payment provider.

[0488] Delivery and assembly services

[0489] The server checks the inventory status of the purchased furniture and automatically adjusts the delivery schedule. The confirmed delivery date and time is notified to the terminal, and the user picks up the furniture on the specified date. The delivery staff arrives and assembles and sets up the furniture according to the user's wishes.

[0490] Specific examples

[0491] The user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates modern-style interior design proposals. The generated design proposals are displayed on the device, allowing the user to change the color of the sofa from red to blue or move the TV stand to the other side of the window. The device then displays the furniture prices and total budget to the user, who then makes the online payment. The server then schedules delivery and assembly services for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[0492] Prompt Sentence Examples

[0493] "Analyze the photo of the room below and suggest some modern interior design ideas. The user wants the sofa to be red and the TV stand to be placed opposite the window."

[0494] As described above, this system allows users to propose and realize interior designs that are efficient and convenient through a series of processes.

[0495] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0496] Step 1: Take and upload a photo

[0497] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. Specifically, they take photos of the entire living room, the area around the sofa, and the position of the TV stand. The input is the captured image data (JPEG format).

[0498] The device temporarily stores the captured photos in its internal memory, after which the user can select the photos they want to upload on the upload screen.

[0499] The terminal provides an upload button. When the user presses the upload button, the selected photo data is sent to the server. The output is the image data sent to the server.

[0500] Step 2: Receiving and analyzing photo data

[0501] The server receives the image data. The input is the image data sent from the terminal.

[0502] The server launches an image analysis module (such as OpenCV or TensorFlow) and analyzes the room's characteristics (dimensions, furniture layout, hue, etc.) from the photo data. Specifically, edge detection and hue distribution calculations are performed. Data processing involves extracting features from the image and identifying the room's dimensions and furniture layout.

[0503] The analysis results (room dimensions, furniture layout, color tone, etc.) are saved as output and used in the next process.

[0504] Step 3: Generate interior design ideas

[0505] The server generates interior design proposals using a generative AI model (such as GPT-3 or DALL-E) based on the analysis results. The input is the analysis results from the image analysis module.

[0506] The server inputs a prompt to the generative AI model, for example, "Please suggest a design for a modern living room. The user wants the sofa to be red, and the TV stand to be placed opposite the window."

[0507] The generative AI model outputs interior design ideas. The output is a generated interior design idea.

[0508] Step 4: Displaying interior design ideas

[0509] The terminal displays the generated interior design proposal on a user interface. The input is the interior design proposal sent from the server.

[0510] The device displays design proposals to the user through an interactive UI, allowing them to freely rotate and zoom in and out of the entire room using 3D rendering.

[0511] The output is an interior design proposal displayed to the user.

[0512] Step 5: Adjust the design

[0513] The user can adjust the color, placement, type, etc. of the furniture based on the displayed design proposal. Input is via an interactive UI on the device.

[0514] The terminal transmits the user's adjustments to the server in real time, and the output is the adjustment data transmitted to the server.

[0515] The server receives the adjustments and generates a new design proposal, which is then sent back to the device.

[0516] The output is a revised interior design proposal.

[0517] Step 6: Budget review and closing

[0518] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user. The input is the furniture information sent from the server.

[0519] The terminal calculates the total budget and presents it to the user. The budget calculation is based on the price information of the selected furniture. The output is the total budget presented to the user.

[0520] The user confirms the budget and moves to the online payment screen. They enter the necessary payment information (credit card number, etc.) and make the payment.

[0521] The server receives the payment information and completes the payment process with the external payment provider. The output is a confirmation of successful payment.

[0522] Step 7: Delivery and assembly service

[0523] The server checks the stock status of the purchased furniture. The stock information is stored in a database and is searched using an SQL query. The input is the user's order information.

[0524] After the server checks the inventory, it automatically adjusts the delivery schedule. It works with the Delivery Administrator API to set the optimal delivery date. The output is the confirmed delivery date and time.

[0525] The server arranges delivery and assembly services, notifies the terminal of the confirmed delivery date and time, and outputs the delivery information notified to the user.

[0526] The user receives the furniture on the scheduled delivery date. The delivery staff arrives at the specified time and assembles and installs the purchased furniture. The output is a confirmation that assembly and installation have been completed.

[0527] (Application example 1)

[0528] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0529] While there are currently support systems that offer a complete range of services from interior design proposals to furniture purchases and installation, there are few ways for users to visually check the actual furniture placement and adjust it in real time. This often leads to problems such as the furniture not matching the image after purchase, making it difficult for users to design their home interiors without stress.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0531] In this invention, the server includes means for acquiring and uploading images of the user's interior, means for analyzing the characteristics of the interior from the uploaded images, means for displaying the generated interior design, means for the user to adjust the displayed design, means for calculating costs and making payments based on the selected furniture and design, means for delivering and assembling the selected furniture, and means for visually displaying the generated design in augmented reality using a smart device. This allows the user to visually check the actual furniture placement and adjust it in real time, easily achieving the optimal interior design.

[0532] "Means for acquiring and uploading images of one's own room" refers to a function that allows a user to take a photo of the room using a smartphone or camera and send the image data to the system.

[0533] "Means for analyzing the interior features from uploaded images" refers to a function in which the system analyzes uploaded image data and extracts features such as the interior dimensions, furniture arrangement, and color hue.

[0534] The "means for displaying the generated interior design" is a function for visually presenting to the user an interior design proposal generated based on the analyzed data.

[0535] "Means for users to adjust the displayed design" refers to a function that provides an interface that allows users to change the furniture placement, color, type, etc. of the presented design proposal.

[0536] "Means for calculating costs and making payments based on the selected furniture and design" is a function that calculates the total cost based on the furniture and design selected by the user and allows payment to be made online.

[0537] "Means for delivering and assembling selected furniture" is a function that arranges for the furniture purchased by the user to be delivered to the specified address and properly assembled and installed.

[0538] "Means for visually displaying the generated design in augmented reality using a smart device" refers to a function that uses devices such as smart glasses or a headset to display the generated interior design superimposed on the actual room, allowing the user to check its layout in real time.

[0539] This invention is a system that allows users to take pictures of their own rooms, propose and adjust interior designs based on the images, and ultimately purchase and install furniture efficiently. Specific embodiments of this system are described below.

[0540] First, the user takes a photo of their room using a device such as a smartphone or tablet. The photo is temporarily saved on the device and then sent to the server by pressing the upload button. At this time, the image data is transferred in an appropriate format.

[0541] The server then receives the uploaded images and uses an image analysis module to automatically analyze characteristics such as the room's dimensions, furniture placement, and color. Based on the analysis results, a generative AI model automatically generates interior design proposals. The output of the generative AI model is adjusted taking into account the user's past preference data and current interior design trends.

[0542] The generated interior design proposals are displayed in real time on smart devices. For example, if the user is wearing smart glasses, the generated design is overlaid on the real room using augmented reality (AR) technology, allowing the user to visually confirm the furniture and design as if they were actually placed in the room.

[0543] If users like the design, they can adjust the color, placement, and type of furniture. These adjustments are made through an interactive UI, with real-time feedback.

[0544] The terminal also displays detailed information about the selected furniture (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The payment process is handled by the server.

[0545] Finally, the server checks the inventory status of the purchased furniture and automatically schedules the delivery. It also arranges for delivery and assembly services and notifies the user of the delivery date and time. The user picks up the furniture on the scheduled delivery date, and the delivery staff assembles and sets it up.

[0546] For example, a user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's smart glasses, and the user can review it and change the color of the sofa from red to blue and move the TV stand to the other side of the window. Finally, the user confirms the furniture price and total budget, and makes an online payment. The delivery staff delivers the furniture on the specified date and assembles it according to the user's wishes.

[0547] An example of input to a generative AI model is the following prompt:

[0548] "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m."

[0549] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0550] Step 1: The user takes a photo of their room using a smartphone or tablet.

[0551] Input: A photo of an interior taken with a smartphone or tablet.

[0552] Output: The captured photos are temporarily saved on the device.

[0553] Specific behavior: The user launches the camera app and takes several photos of the room. If necessary, the app will display guidance on how to take photos properly.

[0554] Step 2: The user uploads a photo to the server.

[0555] Input: The indoor photo taken in step 1.

[0556] Output: Photo data sent to the server by pressing the upload button.

[0557] How it works: The user clicks the upload button in the app, and the selected photos are sent to the server. The photo data is transferred in a standard format such as JPEG.

[0558] Step 3: The server analyzes the uploaded image.

[0559] Input: Photo data of the room sent to the server.

[0560] Output: Analyzed interior features such as dimensions, furniture layout, and color.

[0561] Specific operation: The server uses the image analysis module to analyze the dimensions, furniture arrangement, and color of the photo, and as a result, obtains the interior feature data.

[0562] Step 4: The server uses the generative AI model to generate interior design proposals.

[0563] Input: Analyzed indoor feature data, user's past preference data, current interior trends.

[0564] Output: Generated interior design proposals.

[0565] How it works: The server uses the generative AI model to generate interior design proposals based on a prompt such as "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m." The output design proposals are saved in a digital format.

[0566] Step 5: The device displays the generated interior design proposal to the user.

[0567] Input: Generated interior design proposal.

[0568] Output: Interior design proposal displayed on the user's device.

[0569] How it works: Smart glasses, tablets, and other devices receive the design proposals and use augmented reality (AR) technology to overlay them onto the user's field of vision, allowing the user to view the design proposals through their device.

[0570] Step 6: The user adjusts the design proposal.

[0571] Input: Adjustment instructions from the user (changes to furniture color, placement, type, etc.).

[0572] Output: Adjusted interior design proposal.

[0573] Specific operation: The user operates the device to change the color, arrangement, type, etc. of the furniture in the design proposal. The changes are reflected in real time, and the final design proposal is updated.

[0574] Step 7: The device will display detailed information about the selected furniture and calculate your budget.

[0575] Input: Adjusted interior design proposal.

[0576] Output: Furniture details (price, brand, dimensions, etc.), calculated budget.

[0577] Specific operation: The device displays detailed information about the selected furniture based on the adjusted design proposal. The total cost and individual costs are also calculated and presented to the user.

[0578] Step 8: The user makes an online payment.

[0579] Input: Final interior design proposal and adjusted budget.

[0580] Output: Payment completion notification.

[0581] What happens: The user presses the payment button on the device to complete the online payment. The payment is made through a secure protocol, and a notification is sent to the server upon success.

[0582] Step 9: The server arranges furniture delivery and assembly services.

[0583] Input: Payment completion notification, inventory information.

[0584] Output: Notification of delivery date and time and arrangement of assembly staff.

[0585] Specific operation: The server checks the furniture inventory and schedules the optimal delivery date and assembly service. The delivery date and time are notified to the user's device, and delivery and assembly work are carried out on the specified date.

[0586] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0587] This invention relates to a system that allows for efficient and easy home interior design, and in particular provides more user-friendly interior designs by combining an emotion engine that recognizes the user's emotions and reflects them in design proposals. The program for this system is specifically implemented as follows through interactions between a server, terminals, and users, and the use of the emotion engine.

[0588] Detailed explanation of functions

[0589] 1. Take and upload a photo

[0590] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[0591] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[0592] 2. Interior design proposals

[0593] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0594] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0595] 3. Adjust the design

[0596] The device displays the generated interior design proposals to the user in an interactive UI.

[0597] The user can view the displayed design proposal and adjust the color, placement, type, etc. of the furniture. The device reflects the user's changes in real time and displays the updated design.

[0598] 4. Utilizing the Emotion Engine

[0599] To recognize the user's emotions, the device uses an emotion engine, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[0600] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0601] 5. Budget confirmation and settlement

[0602] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[0603] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0604] 6. Delivery and assembly services

[0605] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0606] The user prepares to receive the furniture on the scheduled delivery date.

[0607] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[0608] Specific examples

[0609] The user takes three photos of their living room and follows the app's instructions to upload them.

[0610] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[0611] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[0612] The device reads the user's facial expression, and the emotion engine recognizes that the user is in a relaxed state, and sends this to the server.

[0613] Based on the data from the emotion engine, the server will suggest more subdued colors and relaxing furniture.

[0614] The user checks the furniture price and total budget and makes an online payment. The server schedules the delivery and assembly service of the purchased furniture and notifies the terminal.

[0615] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[0616] As described above, this system provides users with the proposal and realization of highly convenient interior designs through a series of processes including the emotion engine.

[0617] The processing flow will be explained below.

[0618] Step 1:

[0619] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[0620] Step 2:

[0621] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[0622] Step 3:

[0623] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0624] Step 4:

[0625] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[0626] Step 5:

[0627] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[0628] Step 6:

[0629] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[0630] Step 7:

[0631] The device reads the user's facial expressions and tone of voice, and the emotion engine recognizes their current emotional state. For example, if the user is smiling, the emotion is determined to be "joy."

[0632] Step 8:

[0633] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0634] Step 9:

[0635] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[0636] Step 10:

[0637] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0638] Step 11:

[0639] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0640] Step 12:

[0641] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[0642] Step 13:

[0643] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[0644] Example 2

[0645] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0646] Conventional interior design systems have difficulty proposing personalized designs that reflect the user's emotions. Furthermore, selecting and arranging furniture by oneself can be time-consuming, which can lead to a lack of completeness in the design. Furthermore, the complicated procedures involved in calculating costs and arranging furniture delivery and assembly reduce user convenience.

[0647] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing the characteristics of the space from the transmitted image, a means for automatically generating layout proposals using a generative AI model, and a means for recognizing the user's emotional state and adjusting the proposals based on that. This enables personalized interior design that reflects the user's emotions, and also enables efficient and consistent design adjustments, cost calculations, and furniture delivery and assembly procedures.

[0648] "Image of space" is video information taken by a user to show the situation of his / her specific location.

[0649] "Means of transmission" refers to the technology that transfers images of the space taken by the user to a server using wireless communication or the Internet.

[0650] "Spatial characteristics" refers to information extracted through image analysis, such as spatial dimensions, furniture arrangement, color, and lighting.

[0651] The "means of analysis" refers to technology that automatically extracts spatial characteristics using image analysis modules and artificial intelligence algorithms.

[0652] The "layout proposal" is a plan that proposes the optimal furniture layout and interior design based on the analyzed characteristics of the space.

[0653] The "display means" refers to a technology for visually presenting the generated layout plan to the user, and includes, for example, a mobile app or a web app.

[0654] "Adjustment means" refers to techniques that allow users to interactively change and adapt the displayed layout plan.

[0655] "Cost" refers to the total cost of the selected furniture and interior.

[0656] The "processing means" refers to the technology that calculates the cost of the furniture and interior items selected by the user and allows for online payment.

[0657] "Delivery and assembly means" refers to the technology and services for transporting the selected furniture to the customer's designated location and assembling it on-site.

[0658] A "generative AI model" is an artificial intelligence algorithm with natural language processing and image generation capabilities, and is a model for automatically generating placement plans.

[0659] "Emotional state" refers to the mental and emotional state of the user ascertained by analyzing facial expressions and tone of voice.

[0660] The "means of recognition" is a technology that uses an emotion engine to identify the user's emotional state.

[0661] This invention is a system that proposes personalized interior design taking into account the user's emotions. This system is realized using the interaction of a server, a terminal, and a user. Below, we will explain how data is processed using specific hardware and software.

[0662] First, a user takes multiple photos of their room using a device such as a smartphone or tablet. These photos are temporarily stored on the device, and the user can select one. The user then presses the upload button on the device to send the selected photos to the server. The server then saves the received photo data in an appropriate format (e.g., JPEG or PNG).

[0663] The server then launches an image analysis module to analyze the received photo data. Specifically, image processing libraries such as OpenCV and TensorFlow can be used. The analysis module automatically extracts characteristics such as room dimensions, furniture placement, and color hues. This extracted data is used to create interior design proposals.

[0664] After the analysis is complete, the server uses a generative AI model to automatically generate interior design proposals. Examples of such models include DALL-E and GPT-4. The design generation takes into account not only the analyzed room characteristic data, but also the user's past preference data and current interior trends. An example of a prompt sentence input to the generative AI model is as follows:

[0665] Example prompt sentence:

[0666] Room size: 5m x 6m. Furniture arrangement: Sofa, TV stand, coffee table. Preferred style: Modern. Favorite colors: Blue, gray. Current interior trends: Natural materials, simple design.

[0667] The generated interior design proposal is displayed to the user through the device's interactive user interface (UI). The user can interactively adjust the color, placement, and type of furniture in the displayed design proposal. For example, the user can change the color of a sofa from red to blue, or move a TV stand to the other side of a window. The device then reflects the user's changes in real time and redisplays the updated design proposal.

[0668] The device then uses an emotion engine to analyze the user's facial expressions and tone of voice. The emotion engine uses technologies such as the Emotion API from Microsoft Azure Cognitive Services. The emotional data obtained as a result of the analysis contains information such as whether the user is relaxed or excited. Based on this, the server can further adjust the design proposal. For example, if the user is in a relaxed state, the system will suggest calming colors and relaxing furniture.

[0669] Finally, the terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user confirms the budget and, if there are no problems, makes an online payment. The payment information is sent from the terminal to the server, and the payment is processed. The server then arranges for delivery and assembly services, and notifies the terminal of the detailed delivery date and time.

[0670] The customer prepares to receive the furniture on the specified delivery date. The delivery staff delivers the furniture to the customer's address on the specified date and assembles it on site. The customer checks the completed furniture and evaluates whether the service was satisfactory.

[0671] Through the above process, the system provides users with an easy and efficient way to propose and realize interior designs. By utilizing the emotion engine, more personalized design proposals are provided, improving user satisfaction.

[0672] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0673] Step 1:

[0674] A user launches the camera app on their smartphone or tablet and takes a photo of their room. They take multiple photos of different rooms, such as the living room and bedroom. These photo data are temporarily stored on the device and the user can select one. The input is the captured photo data, and the output is the selected photo data.

[0675] Step 2:

[0676] The device displays a photo upload button, and the user presses the upload button to select a photo. The device converts the selected photo into an appropriate format (e.g., JPEG, PNG) and sends it to the server. The input is the selected photo data, and the output is the photo data sent to the server.

[0677] Step 3:

[0678] The server stores the received photo data in storage. Next, the server launches an image analysis module (e.g., OpenCV, TensorFlow) to analyze the photo data. The analysis module identifies the room's dimensions, furniture layout, color hue, etc. The input is the received photo data, and the output is the analyzed room characteristic data.

[0679] Step 4:

[0680] The server uses the analyzed room characteristic data to launch a generative AI model (e.g., DALL-E, GPT-4) and automatically generate interior design proposals. The generation process also takes into account the user's past preference data and current interior trends. The input is the analyzed room characteristic data and the user's preference data, and the output is the generated interior design proposals.

[0681] Step 5:

[0682] The device displays the generated interior design proposal to the user in an interactive user interface (UI), allowing the user to check the displayed design proposal. The input is the generated interior design proposal, and the output is the design proposal displayed to the user.

[0683] Step 6:

[0684] The user adjusts the design proposal using the UI. For example, the user may change the color of the sofa from red to blue or rearrange the furniture. The device reflects the user's changes in real time and redisplays the updated design proposal. The input is the user's adjustment data, and the output is the updated design proposal.

[0685] Step 7:

[0686] The device uses the built-in camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state. This is done using an emotion engine (e.g., Microsoft Azure's Emotion API). The input is the user's facial expression and voice data, and the output is the recognized emotional state data.

[0687] Step 8:

[0688] The server further adjusts the interior design proposal based on the emotional state data obtained from the emotion engine. For example, if the user is in a relaxed state, it will suggest calming colors and relaxing furniture. The input is the emotional state data, and the output is the adjusted interior design proposal.

[0689] Step 9:

[0690] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the selected furniture data, and the output is the calculated budget data.

[0691] Step 10:

[0692] The user checks the budget and, if there are no problems, presses the online payment button. The terminal sends the payment information to the server, which then processes the payment. The input is the payment information, and the output is payment completion data.

[0693] Step 11:

[0694] The server checks the inventory and sets the delivery date for the purchased furniture. The server arranges the delivery and assembly service and notifies the terminal of the details. The input is the purchased furniture data, and the output is the delivery schedule data.

[0695] Step 12:

[0696] The user receives the furniture on the specified delivery date. The delivery staff brings the furniture to the user's address and assembles it on site. The user checks the completed furniture and rates whether the service was satisfactory. The input is information about the delivery and assembly service, and the output is the user's satisfaction rating data.

[0697] (Application example 2)

[0698] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0699] Conventional interior design systems have the problem of reducing user satisfaction because they propose designs based solely on the physical characteristics of the room and past preference data, without taking the user's emotional state into consideration. In particular, the interior design of autonomous vehicles requires providing a design that is optimal for the user's emotional state while traveling, but this issue has not been adequately resolved.

[0700] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking and uploading a photo of the user's room, means for analyzing characteristics of the room from the uploaded photo, means for displaying generated interior design proposals, means for the user to adjust the generated interior design proposals, means for recognizing the user's emotional state, means for adjusting the interior design proposals based on the emotional data, means for calculating a budget based on the selected furniture and design and making payments, and means for delivering and assembling the selected furniture. This makes it possible to provide an interior design that is optimal for the user's emotional state in real time, thereby improving satisfaction.

[0701] "Means to take and upload photos of your room" is a function that allows you to take photos of your room or the inside of your car using a smartphone or camera and transfer the image data to the system's server.

[0702] "Means for analyzing room characteristics from uploaded photos" is a function for analyzing received photo data and extracting the dimensions, furniture arrangement, color tone, etc. of a room or vehicle interior.

[0703] The "means for displaying the generated interior design proposal" is a function for visually displaying the interior design proposal generated by the server on the user's terminal.

[0704] The "means for the user to adjust the generated interior design proposal" is a function that allows the user to change the color, layout, materials, etc. of the displayed interior design proposal.

[0705] The "means for recognizing the user's emotional state" is a function for analyzing the user's facial expression, tone of voice, etc., to identify the user's emotional state at that time.

[0706] The "means for adjusting interior design proposals based on emotional data" is a function for further optimizing interior design proposals based on the recognized emotional data.

[0707] "Means for calculating a budget and making payments based on the selected furniture and design" is a function that calculates the overall budget based on the furniture and design selected by the user and makes online payments within that budget.

[0708] The "means for delivering and assembling the selected furniture" is a function for delivering the determined furniture to the user and assembling the furniture.

[0709] "Means for automatically generating interior design proposals using a generative AI model based on the analyzed characteristics of a room" is a function that uses a generative AI model to automatically propose interior designs based on the analysis results of a room or car interior.

[0710] "Means for customizing interior design proposals based on the user's past preference data and current interior trends" is a function for customizing interior design proposals taking into account the user's past preference data and the latest interior trends.

[0711] This invention is a system for efficiently and easily customizing the interior design of an autonomous vehicle. In particular, it provides a more user-friendly interior design by combining an emotion engine that recognizes the user's emotional state and reflects it in the design proposal.

[0712] The system program is specifically implemented as follows through the interaction between the server, the terminal, and the user and the use of the emotion engine.

[0713] Taking and uploading photos

[0714] Users can take photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device and can be selected by the user.

[0715] The device provides an upload button to send the user-selected photos to the server, where the photo data is transferred in the appropriate format.

[0716] Interior design proposals

[0717] The server then launches an image analysis module to analyze the received photo data, automatically extracting characteristics such as room or vehicle interior dimensions, furniture placement, and color hues.

[0718] The server uses a generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0719] Design Adjustments

[0720] The device displays the generated interior design proposal to the user in an interactive UI. Using a smartphone or head-mounted display, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture. The device reflects the user's changes in real time and displays the updated design.

[0721] Utilizing the Emotion Engine

[0722] The device uses an emotion engine to recognize the user's emotions, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[0723] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0724] Budget confirmation and payment

[0725] The terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.) and calculates the total budget and presents it to the user.

[0726] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0727] Delivery and assembly services

[0728] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0729] The user prepares to receive the furniture on the scheduled delivery date.

[0730] The delivery staff will deliver the furniture to the user's address at the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[0731] Specific examples

[0732] The user takes three photos of the interior of the autonomous vehicle and uploads them as instructed by the app. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's device. The user changes the color of the sofa from red to blue and moves the TV stand to the opposite side of the window. The device reads the user's facial expression and the emotion engine recognizes that the user is in a relaxed state, and sends this information to the server. Based on the data from the emotion engine, the server suggests more muted colors and more relaxing furniture. The user checks the price and total budget of the furniture and makes an online payment. The server arranges a delivery and assembly service schedule for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[0733] An example of a prompt for the generative AI model is, "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state." In this way, the system can provide an interior design that best suits the user's emotional state in real time, improving satisfaction.

[0734] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0735] Step 1:

[0736] The user takes photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device. The input is a photo of each location, and the output is the photo data stored on the device. At this stage, the user can take multiple photos as needed and select the appropriate photo according to the application's instructions.

[0737] Step 2:

[0738] The device provides an upload button and sends the photos selected by the user to the server. The input here is the photo data selected by the user, and the output is the photo data sent to the server. When the user presses the upload button, the device converts the photo data into an appropriate format and transfers it to the server.

[0739] Step 3:

[0740] The server launches an image analysis module to analyze the received photo data. The input is the photo data stored on the server, and the output is characteristics such as the dimensions of the room or car interior, furniture arrangement, and color hue. The server uses image analysis algorithms to extract these characteristics from the photo data and prepare it for further processing.

[0741] Step 4:

[0742] The server uses a generative AI model to generate interior design proposals based on the analysis results. The input is the analyzed characteristic data of the room or car interior, and the output is an interior design proposal. In particular, the user's past preference data and current interior trends are also taken into consideration. For example, a prompt sentence such as "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state" is input into the generative AI model.

[0743] Step 5:

[0744] The device displays the generated interior design proposal to the user in an interactive UI. The input is the interior design proposal sent from the server, and the output is the design proposal displayed on the device. At this stage, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture.

[0745] Step 6:

[0746] The device uses an emotion engine to recognize the user's emotions. Here, the device analyzes the user's facial expressions and tone of voice to identify their current emotional state. The input is the user's audio and video data, and the output is the analyzed emotional state data.

[0747] Step 7:

[0748] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. The input is the user's emotional state data and the interior design proposal, and the output is the adjusted design proposal. For example, if the user feels like relaxing, the server will suggest calming colors and relaxing furniture.

[0749] Step 8:

[0750] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the adjusted interior design plan and detailed furniture information, and the output is the calculated budget and displayed information.

[0751] Step 9:

[0752] The user checks the budget and makes an online payment. The input is the calculated budget and the user's payment information, and the output is payment confirmation data. The payment details are sent from the terminal to the server, which then processes the payment.

[0753] Step 10:

[0754] The server checks the inventory status of the purchased furniture and sets a delivery date. The server also arranges for delivery and assembly services and notifies the terminal of the delivery date and time. The input is furniture inventory data and the user's desired delivery date and time, and the output is delivery and assembly schedule information.

[0755] Step 11:

[0756] The user prepares to receive the furniture on the scheduled delivery date. The delivery staff delivers the furniture to the user's address on the specified date and time and assembles it. The input is the schedule information and the furniture itself, and the output is a report that the assembly is complete. The user checks the assembled furniture and evaluates whether the service was satisfactory.

[0757] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0758] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0759] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0760] [Third embodiment]

[0761] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0762] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0763] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0764] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0765] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0766] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0767] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0768] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0769] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0770] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0771] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0772] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0773] This invention relates to a system that allows users to efficiently and easily design their home interiors. By taking and uploading photos of their own rooms, the system provides comprehensive support, from interior design proposals to furniture purchases and installation. The program for this system is specifically implemented as follows through interactions between a server, a terminal, and a user.

[0774] Detailed explanation of functions

[0775] 1. Take and upload a photo

[0776] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[0777] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[0778] 2. Interior design proposals

[0779] The server receives the uploaded photo data, and the image analysis module automatically analyzes characteristics such as room dimensions, furniture placement, and color hue.

[0780] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0781] 3. Adjust the design

[0782] The device displays the generated interior design proposals to the user in an interactive UI.

[0783] Users can view the displayed design proposals and adjust the color, placement, and type of furniture, and the system will reflect the user's changes in real time.

[0784] 4. Budget confirmation and settlement

[0785] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[0786] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent to the server and payment processing is carried out.

[0787] 5. Delivery and assembly services

[0788] The server checks the stock status of the purchased furniture and automatically adjusts the delivery schedule.

[0789] The server arranges delivery and assembly services and notifies the terminal of the delivery date and time.

[0790] The user receives the furniture on the scheduled delivery date, and the delivery staff assembles and installs the furniture.

[0791] Specific examples

[0792] The user takes three photos of their living room and follows the app's instructions to upload them.

[0793] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[0794] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[0795] The terminal displays the furniture prices and total budget to the user, who then makes the payment online.

[0796] The server arranges the delivery and assembly service schedule for the purchased furniture and notifies the terminal.

[0797] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[0798] As described above, this system allows users to propose and realize highly convenient interior designs through a series of processes.

[0799] The processing flow will be explained below.

[0800] Step 1:

[0801] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[0802] Step 2:

[0803] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[0804] Step 3:

[0805] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0806] Step 4:

[0807] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[0808] Step 5:

[0809] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[0810] Step 6:

[0811] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[0812] Step 7:

[0813] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[0814] Step 8:

[0815] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0816] Step 9:

[0817] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0818] Step 10:

[0819] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[0820] Step 11:

[0821] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[0822] Example 1

[0823] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0824] Conventional interior design systems require users to manually input room dimensions and furniture layouts, making operation cumbersome and inefficient. Furthermore, if the proposed interior design does not match the user's preferences, readjusting or resubmitting the design is cumbersome, requiring time and effort.

[0825] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0826] In this invention, the server includes: a means for users to take and upload photos of their rooms; a means for the server to analyze the characteristics of the room from the uploaded photos; a means for the server to automatically generate interior design proposals using a generative AI model based on the analyzed characteristics; a means for the terminal to display the generated interior design proposals; a means for the user to adjust the displayed design proposals; a means for the terminal to calculate a budget based on the selected furniture and design and make payments; and a means for the server to deliver and assemble the selected furniture. This allows users to easily receive interior design proposals simply by uploading photos of their rooms, and the generative AI model automatically generates optimal design proposals taking into account the user's past preference data and current interior trends, allowing for adjustments in real time. This significantly reduces the amount of work required and enables efficient and satisfying interior design proposals.

[0827] "User" refers to an individual who uses this system to propose and realize interior designs.

[0828] "Server" refers to the computer system that receives the photo data, analyzes the images, generates interior design suggestions using generative AI models, and arranges for the delivery and assembly of the furniture.

[0829] "Device" refers to the device (e.g., smartphone or tablet) that a user uses to take and upload photos and view and adjust the generated interior design proposals.

[0830] "Photo data" refers to an image file that a user takes of their room and sends from their terminal to the server.

[0831] "Image analysis module" refers to a software module for automatically analyzing room characteristics from uploaded photo data.

[0832] "Generative AI model" refers to an artificial intelligence model used to generate interior design ideas based on analyzed room characteristics.

[0833] "Interior design proposal" refers to a series of design suggestions, such as furniture arrangements and color hues, generated by a generative AI model.

[0834] "User Interface" refers to the interactive screen on the terminal that allows a user to view and adjust interior design proposals.

[0835] "Budget calculation means" refers to a function for calculating the overall budget based on the furniture and design selected by the user.

[0836] "Payment method" refers to the function that allows users to pay for furniture and other items online.

[0837] "Delivery and assembly arrangement means" refers to a function for checking the inventory of furniture purchased by the user and arranging delivery and assembly.

[0838] "User's past preference data" refers to historical data about interior designs and furniture that the user has selected in the past.

[0839] "Interior trends" refers to interior design trends and styles that are currently popular in the market.

[0840] This invention relates to a system that allows users to efficiently and easily design their home interiors. Users can take and upload photos of their rooms, and the system provides comprehensive support from interior design proposals to furniture purchases and installation. This system is specifically implemented through interactions between a server, terminals, and users.

[0841] Hardware and Software

[0842] Server: This is the computer system that serves as the core of the system, and is generally a cloud server or dedicated server. It receives photo data, analyzes images, generates interior design proposals using generative AI models, and arranges for furniture delivery and assembly. For example, OpenCV is used for image analysis, and GPT-3 or DALL-E is used for generative AI models.

[0843] Terminal: A device operated by the user, typically a smartphone or tablet. Photos are taken using the device's camera and uploaded to a server via an application. The device also displays and adjusts interior design proposals, checks budgets, and processes payments.

[0844] User: An individual who uses the system to propose and realize interior designs. Users take photos of their rooms, select and arrange furniture based on the interior design proposals, and finally purchase the furniture.

[0845] Program processing

[0846] Taking and uploading photos

[0847] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. For example, they can take photos of the entire living room, the area around the sofa, and specific locations such as the position of the TV stand. The photos are temporarily saved on the device, and the user can press the upload button to send the selected photos to the server.

[0848] Interior design proposals

[0849] The server analyzes the received photo data using an image analysis module (e.g., OpenCV or TensorFlow). The analysis results include the room's dimensions, furniture layout, color hues, etc. Based on these analysis results, the server generates interior design proposals using a generative AI model (e.g., GPT-3 or DALL-E). This process also takes into account the user's past preference data and current interior trends.

[0850] Design Adjustments

[0851] The device displays the generated interior design proposals in an interactive user interface. The user can adjust the color, placement, and type of furniture based on the displayed design proposals. The system reflects the user's changes in real time, and the server instantly generates a new design proposal based on the changes and returns it to the device for display.

[0852] Budget confirmation and settlement

[0853] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The server receives the payment information and completes the payment process by connecting with an external payment provider.

[0854] Delivery and assembly services

[0855] The server checks the inventory status of the purchased furniture and automatically adjusts the delivery schedule. The confirmed delivery date and time is notified to the terminal, and the user picks up the furniture on the specified date. The delivery staff arrives and assembles and sets up the furniture according to the user's wishes.

[0856] Specific examples

[0857] The user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates modern-style interior design proposals. The generated design proposals are displayed on the device, allowing the user to change the color of the sofa from red to blue or move the TV stand to the other side of the window. The device then displays the furniture prices and total budget to the user, who then makes the online payment. The server then schedules delivery and assembly services for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[0858] Prompt Sentence Examples

[0859] "Analyze the photo of the room below and suggest some modern interior design ideas. The user wants the sofa to be red and the TV stand to be placed opposite the window."

[0860] As described above, this system allows users to propose and realize interior designs that are efficient and convenient through a series of processes.

[0861] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0862] Step 1: Take and upload a photo

[0863] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. Specifically, they take photos of the entire living room, the area around the sofa, and the position of the TV stand. The input is the captured image data (JPEG format).

[0864] The device temporarily stores the captured photos in its internal memory, after which the user can select the photos they want to upload on the upload screen.

[0865] The terminal provides an upload button. When the user presses the upload button, the selected photo data is sent to the server. The output is the image data sent to the server.

[0866] Step 2: Receiving and analyzing photo data

[0867] The server receives the image data. The input is the image data sent from the terminal.

[0868] The server launches an image analysis module (such as OpenCV or TensorFlow) and analyzes the room's characteristics (dimensions, furniture layout, hue, etc.) from the photo data. Specifically, edge detection and hue distribution calculations are performed. Data processing involves extracting features from the image and identifying the room's dimensions and furniture layout.

[0869] The analysis results (room dimensions, furniture layout, color tone, etc.) are saved as output and used in the next process.

[0870] Step 3: Generate interior design ideas

[0871] The server generates interior design proposals using a generative AI model (such as GPT-3 or DALL-E) based on the analysis results. The input is the analysis results from the image analysis module.

[0872] The server inputs a prompt to the generative AI model, for example, "Please suggest a design for a modern living room. The user wants the sofa to be red, and the TV stand to be placed opposite the window."

[0873] The generative AI model outputs interior design ideas. The output is a generated interior design idea.

[0874] Step 4: Displaying interior design ideas

[0875] The terminal displays the generated interior design proposal on a user interface. The input is the interior design proposal sent from the server.

[0876] The device displays design proposals to the user through an interactive UI, allowing them to freely rotate and zoom in and out of the entire room using 3D rendering.

[0877] The output is an interior design proposal displayed to the user.

[0878] Step 5: Adjust the design

[0879] The user can adjust the color, placement, type, etc. of the furniture based on the displayed design proposal. Input is via an interactive UI on the device.

[0880] The terminal transmits the user's adjustments to the server in real time, and the output is the adjustment data transmitted to the server.

[0881] The server receives the adjustments and generates a new design proposal, which is then sent back to the device.

[0882] The output is a revised interior design proposal.

[0883] Step 6: Budget review and closing

[0884] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user. The input is the furniture information sent from the server.

[0885] The terminal calculates the total budget and presents it to the user. The budget calculation is based on the price information of the selected furniture. The output is the total budget presented to the user.

[0886] The user confirms the budget and moves to the online payment screen. They enter the necessary payment information (credit card number, etc.) and make the payment.

[0887] The server receives the payment information and completes the payment process with the external payment provider. The output is a confirmation of successful payment.

[0888] Step 7: Delivery and assembly service

[0889] The server checks the stock status of the purchased furniture. The stock information is stored in a database and is searched using an SQL query. The input is the user's order information.

[0890] After the server checks the inventory, it automatically adjusts the delivery schedule. It works with the Delivery Administrator API to set the optimal delivery date. The output is the confirmed delivery date and time.

[0891] The server arranges delivery and assembly services, notifies the terminal of the confirmed delivery date and time, and outputs the delivery information notified to the user.

[0892] The user receives the furniture on the scheduled delivery date. The delivery staff arrives at the specified time and assembles and installs the purchased furniture. The output is a confirmation that assembly and installation have been completed.

[0893] (Application example 1)

[0894] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0895] While there are currently support systems that offer a complete range of services from interior design proposals to furniture purchases and installation, there are few ways for users to visually check the actual furniture placement and adjust it in real time. This often leads to problems such as the furniture not matching the image after purchase, making it difficult for users to design their home interiors without stress.

[0896] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0897] In this invention, the server includes means for acquiring and uploading images of the user's interior, means for analyzing the characteristics of the interior from the uploaded images, means for displaying the generated interior design, means for the user to adjust the displayed design, means for calculating costs and making payments based on the selected furniture and design, means for delivering and assembling the selected furniture, and means for visually displaying the generated design in augmented reality using a smart device. This allows the user to visually check the actual furniture placement and adjust it in real time, easily achieving the optimal interior design.

[0898] "Means for acquiring and uploading images of one's own room" refers to a function that allows a user to take a photo of the room using a smartphone or camera and send the image data to the system.

[0899] "Means for analyzing the interior features from uploaded images" refers to a function in which the system analyzes uploaded image data and extracts features such as the interior dimensions, furniture arrangement, and color hue.

[0900] The "means for displaying the generated interior design" is a function for visually presenting to the user an interior design proposal generated based on the analyzed data.

[0901] "Means for users to adjust the displayed design" refers to a function that provides an interface that allows users to change the furniture placement, color, type, etc. of the presented design proposal.

[0902] "Means for calculating costs and making payments based on the selected furniture and design" is a function that calculates the total cost based on the furniture and design selected by the user and allows payment to be made online.

[0903] "Means for delivering and assembling selected furniture" is a function that arranges for the furniture purchased by the user to be delivered to the specified address and properly assembled and installed.

[0904] "Means for visually displaying the generated design in augmented reality using a smart device" refers to a function that uses devices such as smart glasses or a headset to display the generated interior design superimposed on the actual room, allowing the user to check its layout in real time.

[0905] This invention is a system that allows users to take pictures of their own rooms, propose and adjust interior designs based on the images, and ultimately purchase and install furniture efficiently. Specific embodiments of this system are described below.

[0906] First, the user takes a photo of their room using a device such as a smartphone or tablet. The photo is temporarily saved on the device and then sent to the server by pressing the upload button. At this time, the image data is transferred in an appropriate format.

[0907] The server then receives the uploaded images and uses an image analysis module to automatically analyze characteristics such as the room's dimensions, furniture placement, and color. Based on the analysis results, a generative AI model automatically generates interior design proposals. The output of the generative AI model is adjusted taking into account the user's past preference data and current interior design trends.

[0908] The generated interior design proposals are displayed in real time on smart devices. For example, if the user is wearing smart glasses, the generated design is overlaid on the real room using augmented reality (AR) technology, allowing the user to visually confirm the furniture and design as if they were actually placed in the room.

[0909] If users like the design, they can adjust the color, placement, and type of furniture. These adjustments are made through an interactive UI, with real-time feedback.

[0910] The terminal also displays detailed information about the selected furniture (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The payment process is handled by the server.

[0911] Finally, the server checks the inventory status of the purchased furniture and automatically schedules the delivery. It also arranges for delivery and assembly services and notifies the user of the delivery date and time. The user picks up the furniture on the scheduled delivery date, and the delivery staff assembles and sets it up.

[0912] For example, a user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's smart glasses, and the user can review it and change the color of the sofa from red to blue and move the TV stand to the other side of the window. Finally, the user confirms the furniture price and total budget, and makes an online payment. The delivery staff delivers the furniture on the specified date and assembles it according to the user's wishes.

[0913] An example of input to a generative AI model is the following prompt:

[0914] "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m."

[0915] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0916] Step 1: The user takes a photo of their room using a smartphone or tablet.

[0917] Input: A photo of an interior taken with a smartphone or tablet.

[0918] Output: The captured photos are temporarily saved on the device.

[0919] Specific behavior: The user launches the camera app and takes several photos of the room. If necessary, the app will display guidance on how to take photos properly.

[0920] Step 2: The user uploads a photo to the server.

[0921] Input: The indoor photo taken in step 1.

[0922] Output: Photo data sent to the server by pressing the upload button.

[0923] How it works: The user clicks the upload button in the app, and the selected photos are sent to the server. The photo data is transferred in a standard format such as JPEG.

[0924] Step 3: The server analyzes the uploaded image.

[0925] Input: Photo data of the room sent to the server.

[0926] Output: Analyzed interior features such as dimensions, furniture layout, and color.

[0927] Specific operation: The server uses the image analysis module to analyze the dimensions, furniture arrangement, and color of the photo, and as a result, obtains the interior feature data.

[0928] Step 4: The server uses the generative AI model to generate interior design proposals.

[0929] Input: Analyzed indoor feature data, user's past preference data, current interior trends.

[0930] Output: Generated interior design proposals.

[0931] How it works: The server uses the generative AI model to generate interior design proposals based on a prompt such as "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m." The output design proposals are saved in a digital format.

[0932] Step 5: The device displays the generated interior design proposal to the user.

[0933] Input: Generated interior design proposal.

[0934] Output: Interior design proposal displayed on the user's device.

[0935] How it works: Smart glasses, tablets, and other devices receive the design proposals and use augmented reality (AR) technology to overlay them onto the user's field of vision, allowing the user to view the design proposals through their device.

[0936] Step 6: The user adjusts the design proposal.

[0937] Input: Adjustment instructions from the user (changes to furniture color, placement, type, etc.).

[0938] Output: Adjusted interior design proposal.

[0939] Specific operation: The user operates the device to change the color, arrangement, type, etc. of the furniture in the design proposal. The changes are reflected in real time, and the final design proposal is updated.

[0940] Step 7: The device will display detailed information about the selected furniture and calculate your budget.

[0941] Input: Adjusted interior design proposal.

[0942] Output: Furniture details (price, brand, dimensions, etc.), calculated budget.

[0943] Specific operation: The device displays detailed information about the selected furniture based on the adjusted design proposal. The total cost and individual costs are also calculated and presented to the user.

[0944] Step 8: The user makes an online payment.

[0945] Input: Final interior design proposal and adjusted budget.

[0946] Output: Payment completion notification.

[0947] What happens: The user presses the payment button on the device to complete the online payment. The payment is made through a secure protocol, and a notification is sent to the server upon success.

[0948] Step 9: The server arranges furniture delivery and assembly services.

[0949] Input: Payment completion notification, inventory information.

[0950] Output: Notification of delivery date and time and arrangement of assembly staff.

[0951] Specific operation: The server checks the furniture inventory and schedules the optimal delivery date and assembly service. The delivery date and time are notified to the user's device, and delivery and assembly work are carried out on the specified date.

[0952] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0953] This invention relates to a system that allows for efficient and easy home interior design, and in particular provides more user-friendly interior designs by combining an emotion engine that recognizes the user's emotions and reflects them in design proposals. The program for this system is specifically implemented as follows through interactions between a server, terminals, and users, and the use of the emotion engine.

[0954] Detailed explanation of functions

[0955] 1. Take and upload a photo

[0956] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[0957] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[0958] 2. Interior design proposals

[0959] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0960] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[0961] 3. Adjust the design

[0962] The device displays the generated interior design proposals to the user in an interactive UI.

[0963] The user can view the displayed design proposal and adjust the color, placement, type, etc. of the furniture. The device reflects the user's changes in real time and displays the updated design.

[0964] 4. Utilizing the Emotion Engine

[0965] To recognize the user's emotions, the device uses an emotion engine, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[0966] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[0967] 5. Budget confirmation and settlement

[0968] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[0969] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[0970] 6. Delivery and assembly services

[0971] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[0972] The user prepares to receive the furniture on the scheduled delivery date.

[0973] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[0974] Specific examples

[0975] The user takes three photos of their living room and follows the app's instructions to upload them.

[0976] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[0977] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[0978] The device reads the user's facial expression, and the emotion engine recognizes that the user is in a relaxed state, and sends this to the server.

[0979] Based on the data from the emotion engine, the server will suggest more subdued colors and relaxing furniture.

[0980] The user checks the furniture price and total budget and makes an online payment. The server schedules the delivery and assembly service of the purchased furniture and notifies the terminal.

[0981] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[0982] As described above, this system provides users with the proposal and realization of highly convenient interior designs through a series of processes including the emotion engine.

[0983] The processing flow will be explained below.

[0984] Step 1:

[0985] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[0986] Step 2:

[0987] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[0988] Step 3:

[0989] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[0990] Step 4:

[0991] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[0992] Step 5:

[0993] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[0994] Step 6:

[0995] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[0996] Step 7:

[0997] The device reads the user's facial expressions and tone of voice, and the emotion engine recognizes their current emotional state. For example, if the user is smiling, the emotion is determined to be "joy."

[0998] Step 8:

[0999] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[1000] Step 9:

[1001] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[1002] Step 10:

[1003] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[1004] Step 11:

[1005] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[1006] Step 12:

[1007] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[1008] Step 13:

[1009] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[1010] Example 2

[1011] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1012] Conventional interior design systems have difficulty proposing personalized designs that reflect the user's emotions. Furthermore, selecting and arranging furniture by oneself can be time-consuming, which can lead to a lack of completeness in the design. Furthermore, the complicated procedures involved in calculating costs and arranging furniture delivery and assembly reduce user convenience.

[1013] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing the characteristics of the space from the transmitted image, a means for automatically generating layout proposals using a generative AI model, and a means for recognizing the user's emotional state and adjusting the proposals based on that. This enables personalized interior design that reflects the user's emotions, and also enables efficient and consistent design adjustments, cost calculations, and furniture delivery and assembly procedures.

[1014] "Image of space" is video information taken by a user to show the situation of his / her specific location.

[1015] "Means of transmission" refers to the technology that transfers images of the space taken by the user to a server using wireless communication or the Internet.

[1016] "Spatial characteristics" refers to information extracted through image analysis, such as spatial dimensions, furniture arrangement, color, and lighting.

[1017] The "means of analysis" refers to technology that automatically extracts spatial characteristics using image analysis modules and artificial intelligence algorithms.

[1018] The "layout proposal" is a plan that proposes the optimal furniture layout and interior design based on the analyzed characteristics of the space.

[1019] The "display means" refers to a technology for visually presenting the generated layout plan to the user, and includes, for example, a mobile app or a web app.

[1020] "Adjustment means" refers to techniques that allow users to interactively change and adapt the displayed layout plan.

[1021] "Cost" refers to the total cost of the selected furniture and interior.

[1022] The "processing means" refers to the technology that calculates the cost of the furniture and interior items selected by the user and allows for online payment.

[1023] "Delivery and assembly means" refers to the technology and services for transporting the selected furniture to the customer's designated location and assembling it on-site.

[1024] A "generative AI model" is an artificial intelligence algorithm with natural language processing and image generation capabilities, and is a model for automatically generating placement plans.

[1025] "Emotional state" refers to the mental and emotional state of the user ascertained by analyzing facial expressions and tone of voice.

[1026] The "means of recognition" is a technology that uses an emotion engine to identify the user's emotional state.

[1027] This invention is a system that proposes personalized interior design taking into account the user's emotions. This system is realized using the interaction of a server, a terminal, and a user. Below, we will explain how data is processed using specific hardware and software.

[1028] First, a user takes multiple photos of their room using a device such as a smartphone or tablet. These photos are temporarily stored on the device, and the user can select one. The user then presses the upload button on the device to send the selected photos to the server. The server then saves the received photo data in an appropriate format (e.g., JPEG or PNG).

[1029] The server then launches an image analysis module to analyze the received photo data. Specifically, image processing libraries such as OpenCV and TensorFlow can be used. The analysis module automatically extracts characteristics such as room dimensions, furniture placement, and color hues. This extracted data is used to create interior design proposals.

[1030] After the analysis is complete, the server uses a generative AI model to automatically generate interior design proposals. Examples of such models include DALL-E and GPT-4. The design generation takes into account not only the analyzed room characteristic data, but also the user's past preference data and current interior trends. An example of a prompt sentence input to the generative AI model is as follows:

[1031] Example prompt sentence:

[1032] Room size: 5m x 6m. Furniture arrangement: Sofa, TV stand, coffee table. Preferred style: Modern. Favorite colors: Blue, gray. Current interior trends: Natural materials, simple design.

[1033] The generated interior design proposal is displayed to the user through the device's interactive user interface (UI). The user can interactively adjust the color, placement, and type of furniture in the displayed design proposal. For example, the user can change the color of a sofa from red to blue, or move a TV stand to the other side of a window. The device then reflects the user's changes in real time and redisplays the updated design proposal.

[1034] The device then uses an emotion engine to analyze the user's facial expressions and tone of voice. The emotion engine uses technologies such as the Emotion API from Microsoft Azure Cognitive Services. The emotional data obtained as a result of the analysis contains information such as whether the user is relaxed or excited. Based on this, the server can further adjust the design proposal. For example, if the user is in a relaxed state, the system will suggest calming colors and relaxing furniture.

[1035] Finally, the terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user confirms the budget and, if there are no problems, makes an online payment. The payment information is sent from the terminal to the server, and the payment is processed. The server then arranges for delivery and assembly services, and notifies the terminal of the detailed delivery date and time.

[1036] The customer prepares to receive the furniture on the specified delivery date. The delivery staff delivers the furniture to the customer's address on the specified date and assembles it on site. The customer checks the completed furniture and evaluates whether the service was satisfactory.

[1037] Through the above process, the system provides users with an easy and efficient way to propose and realize interior designs. By utilizing the emotion engine, more personalized design proposals are provided, improving user satisfaction.

[1038] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1039] Step 1:

[1040] A user launches the camera app on their smartphone or tablet and takes a photo of their room. They take multiple photos of different rooms, such as the living room and bedroom. These photo data are temporarily stored on the device and the user can select one. The input is the captured photo data, and the output is the selected photo data.

[1041] Step 2:

[1042] The device displays a photo upload button, and the user presses the upload button to select a photo. The device converts the selected photo into an appropriate format (e.g., JPEG, PNG) and sends it to the server. The input is the selected photo data, and the output is the photo data sent to the server.

[1043] Step 3:

[1044] The server stores the received photo data in storage. Next, the server launches an image analysis module (e.g., OpenCV, TensorFlow) to analyze the photo data. The analysis module identifies the room's dimensions, furniture layout, color hue, etc. The input is the received photo data, and the output is the analyzed room characteristic data.

[1045] Step 4:

[1046] The server uses the analyzed room characteristic data to launch a generative AI model (e.g., DALL-E, GPT-4) and automatically generate interior design proposals. The generation process also takes into account the user's past preference data and current interior trends. The input is the analyzed room characteristic data and the user's preference data, and the output is the generated interior design proposals.

[1047] Step 5:

[1048] The device displays the generated interior design proposal to the user in an interactive user interface (UI), allowing the user to check the displayed design proposal. The input is the generated interior design proposal, and the output is the design proposal displayed to the user.

[1049] Step 6:

[1050] The user adjusts the design proposal using the UI. For example, the user may change the color of the sofa from red to blue or rearrange the furniture. The device reflects the user's changes in real time and redisplays the updated design proposal. The input is the user's adjustment data, and the output is the updated design proposal.

[1051] Step 7:

[1052] The device uses the built-in camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state. This is done using an emotion engine (e.g., Microsoft Azure's Emotion API). The input is the user's facial expression and voice data, and the output is the recognized emotional state data.

[1053] Step 8:

[1054] The server further adjusts the interior design proposal based on the emotional state data obtained from the emotion engine. For example, if the user is in a relaxed state, it will suggest calming colors and relaxing furniture. The input is the emotional state data, and the output is the adjusted interior design proposal.

[1055] Step 9:

[1056] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the selected furniture data, and the output is the calculated budget data.

[1057] Step 10:

[1058] The user checks the budget and, if there are no problems, presses the online payment button. The terminal sends the payment information to the server, which then processes the payment. The input is the payment information, and the output is payment completion data.

[1059] Step 11:

[1060] The server checks the inventory and sets the delivery date for the purchased furniture. The server arranges the delivery and assembly service and notifies the terminal of the details. The input is the purchased furniture data, and the output is the delivery schedule data.

[1061] Step 12:

[1062] The user receives the furniture on the specified delivery date. The delivery staff brings the furniture to the user's address and assembles it on site. The user checks the completed furniture and rates whether the service was satisfactory. The input is information about the delivery and assembly service, and the output is the user's satisfaction rating data.

[1063] (Application example 2)

[1064] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1065] Conventional interior design systems have the problem of reducing user satisfaction because they propose designs based solely on the physical characteristics of the room and past preference data, without taking the user's emotional state into consideration. In particular, the interior design of autonomous vehicles requires providing a design that is optimal for the user's emotional state while traveling, but this issue has not been adequately resolved.

[1066] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking and uploading a photo of the user's room, means for analyzing characteristics of the room from the uploaded photo, means for displaying generated interior design proposals, means for the user to adjust the generated interior design proposals, means for recognizing the user's emotional state, means for adjusting the interior design proposals based on the emotional data, means for calculating a budget based on the selected furniture and design and making payments, and means for delivering and assembling the selected furniture. This makes it possible to provide an interior design that is optimal for the user's emotional state in real time, thereby improving satisfaction.

[1067] "Means to take and upload photos of your room" is a function that allows you to take photos of your room or the inside of your car using a smartphone or camera and transfer the image data to the system's server.

[1068] "Means for analyzing room characteristics from uploaded photos" is a function for analyzing received photo data and extracting the dimensions, furniture arrangement, color tone, etc. of a room or vehicle interior.

[1069] The "means for displaying the generated interior design proposal" is a function for visually displaying the interior design proposal generated by the server on the user's terminal.

[1070] The "means for the user to adjust the generated interior design proposal" is a function that allows the user to change the color, layout, materials, etc. of the displayed interior design proposal.

[1071] The "means for recognizing the user's emotional state" is a function for analyzing the user's facial expression, tone of voice, etc., to identify the user's emotional state at that time.

[1072] The "means for adjusting interior design proposals based on emotional data" is a function for further optimizing interior design proposals based on the recognized emotional data.

[1073] "Means for calculating a budget and making payments based on the selected furniture and design" is a function that calculates the overall budget based on the furniture and design selected by the user and makes online payments within that budget.

[1074] The "means for delivering and assembling the selected furniture" is a function for delivering the determined furniture to the user and assembling the furniture.

[1075] "Means for automatically generating interior design proposals using a generative AI model based on the analyzed characteristics of a room" is a function that uses a generative AI model to automatically propose interior designs based on the analysis results of a room or car interior.

[1076] "Means for customizing interior design proposals based on the user's past preference data and current interior trends" is a function for customizing interior design proposals taking into account the user's past preference data and the latest interior trends.

[1077] This invention is a system for efficiently and easily customizing the interior design of an autonomous vehicle. In particular, it provides a more user-friendly interior design by combining an emotion engine that recognizes the user's emotional state and reflects it in the design proposal.

[1078] The system program is specifically implemented as follows through the interaction between the server, the terminal, and the user and the use of the emotion engine.

[1079] Taking and uploading photos

[1080] Users can take photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device and can be selected by the user.

[1081] The device provides an upload button to send the user-selected photos to the server, where the photo data is transferred in the appropriate format.

[1082] Interior design proposals

[1083] The server then launches an image analysis module to analyze the received photo data, automatically extracting characteristics such as room or vehicle interior dimensions, furniture placement, and color hues.

[1084] The server uses a generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[1085] Design Adjustments

[1086] The device displays the generated interior design proposal to the user in an interactive UI. Using a smartphone or head-mounted display, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture. The device reflects the user's changes in real time and displays the updated design.

[1087] Utilizing the Emotion Engine

[1088] The device uses an emotion engine to recognize the user's emotions, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[1089] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[1090] Budget confirmation and payment

[1091] The terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.) and calculates the total budget and presents it to the user.

[1092] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[1093] Delivery and assembly services

[1094] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[1095] The user prepares to receive the furniture on the scheduled delivery date.

[1096] The delivery staff will deliver the furniture to the user's address at the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[1097] Specific examples

[1098] The user takes three photos of the interior of the autonomous vehicle and uploads them as instructed by the app. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's device. The user changes the color of the sofa from red to blue and moves the TV stand to the opposite side of the window. The device reads the user's facial expression and the emotion engine recognizes that the user is in a relaxed state, and sends this information to the server. Based on the data from the emotion engine, the server suggests more muted colors and more relaxing furniture. The user checks the price and total budget of the furniture and makes an online payment. The server arranges a delivery and assembly service schedule for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[1099] An example of a prompt for the generative AI model is, "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state." In this way, the system can provide an interior design that best suits the user's emotional state in real time, improving satisfaction.

[1100] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1101] Step 1:

[1102] The user takes photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device. The input is a photo of each location, and the output is the photo data stored on the device. At this stage, the user can take multiple photos as needed and select the appropriate photo according to the application's instructions.

[1103] Step 2:

[1104] The device provides an upload button and sends the photos selected by the user to the server. The input here is the photo data selected by the user, and the output is the photo data sent to the server. When the user presses the upload button, the device converts the photo data into an appropriate format and transfers it to the server.

[1105] Step 3:

[1106] The server launches an image analysis module to analyze the received photo data. The input is the photo data stored on the server, and the output is characteristics such as the dimensions of the room or car interior, furniture arrangement, and color hue. The server uses image analysis algorithms to extract these characteristics from the photo data and prepare it for further processing.

[1107] Step 4:

[1108] The server uses a generative AI model to generate interior design proposals based on the analysis results. The input is the analyzed characteristic data of the room or car interior, and the output is an interior design proposal. In particular, the user's past preference data and current interior trends are also taken into consideration. For example, a prompt sentence such as "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state" is input into the generative AI model.

[1109] Step 5:

[1110] The device displays the generated interior design proposal to the user in an interactive UI. The input is the interior design proposal sent from the server, and the output is the design proposal displayed on the device. At this stage, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture.

[1111] Step 6:

[1112] The device uses an emotion engine to recognize the user's emotions. Here, the device analyzes the user's facial expressions and tone of voice to identify their current emotional state. The input is the user's audio and video data, and the output is the analyzed emotional state data.

[1113] Step 7:

[1114] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. The input is the user's emotional state data and the interior design proposal, and the output is the adjusted design proposal. For example, if the user feels like relaxing, the server will suggest calming colors and relaxing furniture.

[1115] Step 8:

[1116] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the adjusted interior design plan and detailed furniture information, and the output is the calculated budget and displayed information.

[1117] Step 9:

[1118] The user checks the budget and makes an online payment. The input is the calculated budget and the user's payment information, and the output is payment confirmation data. The payment details are sent from the terminal to the server, which then processes the payment.

[1119] Step 10:

[1120] The server checks the inventory status of the purchased furniture and sets a delivery date. The server also arranges for delivery and assembly services and notifies the terminal of the delivery date and time. The input is furniture inventory data and the user's desired delivery date and time, and the output is delivery and assembly schedule information.

[1121] Step 11:

[1122] The user prepares to receive the furniture on the scheduled delivery date. The delivery staff delivers the furniture to the user's address on the specified date and time and assembles it. The input is the schedule information and the furniture itself, and the output is a report that the assembly is complete. The user checks the assembled furniture and evaluates whether the service was satisfactory.

[1123] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1124] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1125] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1126] [Fourth embodiment]

[1127] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1128] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1129] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1130] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1131] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1132] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1133] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1134] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1135] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1136] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1137] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1138] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1139] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1140] This invention relates to a system that allows users to efficiently and easily design their home interiors. By taking and uploading photos of their own rooms, the system provides comprehensive support, from interior design proposals to furniture purchases and installation. The program for this system is specifically implemented as follows through interactions between a server, a terminal, and a user.

[1141] Detailed explanation of functions

[1142] 1. Take and upload a photo

[1143] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[1144] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[1145] 2. Interior design proposals

[1146] The server receives the uploaded photo data, and the image analysis module automatically analyzes characteristics such as room dimensions, furniture placement, and color hue.

[1147] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[1148] 3. Adjust the design

[1149] The device displays the generated interior design proposals to the user in an interactive UI.

[1150] Users can view the displayed design proposals and adjust the color, placement, and type of furniture, and the system will reflect the user's changes in real time.

[1151] 4. Budget confirmation and settlement

[1152] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[1153] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent to the server and payment processing is carried out.

[1154] 5. Delivery and assembly services

[1155] The server checks the stock status of the purchased furniture and automatically adjusts the delivery schedule.

[1156] The server arranges delivery and assembly services and notifies the terminal of the delivery date and time.

[1157] The user receives the furniture on the scheduled delivery date, and the delivery staff assembles and installs the furniture.

[1158] Specific examples

[1159] The user takes three photos of their living room and follows the app's instructions to upload them.

[1160] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[1161] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[1162] The terminal displays the furniture prices and total budget to the user, who then makes the payment online.

[1163] The server arranges the delivery and assembly service schedule for the purchased furniture and notifies the terminal.

[1164] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[1165] As described above, this system allows users to propose and realize highly convenient interior designs through a series of processes.

[1166] The processing flow will be explained below.

[1167] Step 1:

[1168] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[1169] Step 2:

[1170] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[1171] Step 3:

[1172] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[1173] Step 4:

[1174] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[1175] Step 5:

[1176] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[1177] Step 6:

[1178] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[1179] Step 7:

[1180] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[1181] Step 8:

[1182] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[1183] Step 9:

[1184] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[1185] Step 10:

[1186] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[1187] Step 11:

[1188] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[1189] Example 1

[1190] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1191] Conventional interior design systems require users to manually input room dimensions and furniture layouts, making operation cumbersome and inefficient. Furthermore, if the proposed interior design does not match the user's preferences, readjusting or resubmitting the design is cumbersome, requiring time and effort.

[1192] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1193] In this invention, the server includes: a means for users to take and upload photos of their rooms; a means for the server to analyze the characteristics of the room from the uploaded photos; a means for the server to automatically generate interior design proposals using a generative AI model based on the analyzed characteristics; a means for the terminal to display the generated interior design proposals; a means for the user to adjust the displayed design proposals; a means for the terminal to calculate a budget based on the selected furniture and design and make payments; and a means for the server to deliver and assemble the selected furniture. This allows users to easily receive interior design proposals simply by uploading photos of their rooms, and the generative AI model automatically generates optimal design proposals taking into account the user's past preference data and current interior trends, allowing for adjustments in real time. This significantly reduces the amount of work required and enables efficient and satisfying interior design proposals.

[1194] "User" refers to an individual who uses this system to propose and realize interior designs.

[1195] "Server" refers to the computer system that receives the photo data, analyzes the images, generates interior design suggestions using generative AI models, and arranges for the delivery and assembly of the furniture.

[1196] "Device" refers to the device (e.g., smartphone or tablet) that a user uses to take and upload photos and view and adjust the generated interior design proposals.

[1197] "Photo data" refers to an image file that a user takes of their room and sends from their terminal to the server.

[1198] "Image analysis module" refers to a software module for automatically analyzing room characteristics from uploaded photo data.

[1199] "Generative AI model" refers to an artificial intelligence model used to generate interior design ideas based on analyzed room characteristics.

[1200] "Interior design proposal" refers to a series of design suggestions, such as furniture arrangements and color hues, generated by a generative AI model.

[1201] "User Interface" refers to the interactive screen on the terminal that allows a user to view and adjust interior design proposals.

[1202] "Budget calculation means" refers to a function for calculating the overall budget based on the furniture and design selected by the user.

[1203] "Payment method" refers to the function that allows users to pay for furniture and other items online.

[1204] "Delivery and assembly arrangement means" refers to a function for checking the inventory of furniture purchased by the user and arranging delivery and assembly.

[1205] "User's past preference data" refers to historical data about interior designs and furniture that the user has selected in the past.

[1206] "Interior trends" refers to interior design trends and styles that are currently popular in the market.

[1207] This invention relates to a system that allows users to efficiently and easily design their home interiors. Users can take and upload photos of their rooms, and the system provides comprehensive support from interior design proposals to furniture purchases and installation. This system is specifically implemented through interactions between a server, terminals, and users.

[1208] Hardware and Software

[1209] Server: This is the computer system that serves as the core of the system, and is generally a cloud server or dedicated server. It receives photo data, analyzes images, generates interior design proposals using generative AI models, and arranges for furniture delivery and assembly. For example, OpenCV is used for image analysis, and GPT-3 or DALL-E is used for generative AI models.

[1210] Terminal: A device operated by the user, typically a smartphone or tablet. Photos are taken using the device's camera and uploaded to a server via an application. The device also displays and adjusts interior design proposals, checks budgets, and processes payments.

[1211] User: An individual who uses the system to propose and realize interior designs. Users take photos of their rooms, select and arrange furniture based on the interior design proposals, and finally purchase the furniture.

[1212] Program processing

[1213] Taking and uploading photos

[1214] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. For example, they can take photos of the entire living room, the area around the sofa, and specific locations such as the position of the TV stand. The photos are temporarily saved on the device, and the user can press the upload button to send the selected photos to the server.

[1215] Interior design proposals

[1216] The server analyzes the received photo data using an image analysis module (e.g., OpenCV or TensorFlow). The analysis results include the room's dimensions, furniture layout, color hues, etc. Based on these analysis results, the server generates interior design proposals using a generative AI model (e.g., GPT-3 or DALL-E). This process also takes into account the user's past preference data and current interior trends.

[1217] Design Adjustments

[1218] The device displays the generated interior design proposals in an interactive user interface. The user can adjust the color, placement, and type of furniture based on the displayed design proposals. The system reflects the user's changes in real time, and the server instantly generates a new design proposal based on the changes and returns it to the device for display.

[1219] Budget confirmation and settlement

[1220] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The server receives the payment information and completes the payment process by connecting with an external payment provider.

[1221] Delivery and assembly services

[1222] The server checks the inventory status of the purchased furniture and automatically adjusts the delivery schedule. The confirmed delivery date and time is notified to the terminal, and the user picks up the furniture on the specified date. The delivery staff arrives and assembles and sets up the furniture according to the user's wishes.

[1223] Specific examples

[1224] The user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates modern-style interior design proposals. The generated design proposals are displayed on the device, allowing the user to change the color of the sofa from red to blue or move the TV stand to the other side of the window. The device then displays the furniture prices and total budget to the user, who then makes the online payment. The server then schedules delivery and assembly services for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[1225] Prompt Sentence Examples

[1226] "Analyze the photo of the room below and suggest some modern interior design ideas. The user wants the sofa to be red and the TV stand to be placed opposite the window."

[1227] As described above, this system allows users to propose and realize interior designs that are efficient and convenient through a series of processes.

[1228] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1229] Step 1: Take and upload a photo

[1230] The user launches the camera app on their smartphone or tablet and takes multiple photos of various locations in their room. Specifically, they take photos of the entire living room, the area around the sofa, and the position of the TV stand. The input is the captured image data (JPEG format).

[1231] The device temporarily stores the captured photos in its internal memory, after which the user can select the photos they want to upload on the upload screen.

[1232] The terminal provides an upload button. When the user presses the upload button, the selected photo data is sent to the server. The output is the image data sent to the server.

[1233] Step 2: Receiving and analyzing photo data

[1234] The server receives the image data. The input is the image data sent from the terminal.

[1235] The server launches an image analysis module (such as OpenCV or TensorFlow) and analyzes the room's characteristics (dimensions, furniture layout, hue, etc.) from the photo data. Specifically, edge detection and hue distribution calculations are performed. Data processing involves extracting features from the image and identifying the room's dimensions and furniture layout.

[1236] The analysis results (room dimensions, furniture layout, color tone, etc.) are saved as output and used in the next process.

[1237] Step 3: Generate interior design ideas

[1238] The server generates interior design proposals using a generative AI model (such as GPT-3 or DALL-E) based on the analysis results. The input is the analysis results from the image analysis module.

[1239] The server inputs a prompt to the generative AI model, for example, "Please suggest a design for a modern living room. The user wants the sofa to be red, and the TV stand to be placed opposite the window."

[1240] The generative AI model outputs interior design ideas. The output is a generated interior design idea.

[1241] Step 4: Displaying interior design ideas

[1242] The terminal displays the generated interior design proposal on a user interface. The input is the interior design proposal sent from the server.

[1243] The device displays design proposals to the user through an interactive UI, allowing them to freely rotate and zoom in and out of the entire room using 3D rendering.

[1244] The output is an interior design proposal displayed to the user.

[1245] Step 5: Adjust the design

[1246] The user can adjust the color, placement, type, etc. of the furniture based on the displayed design proposal. Input is via an interactive UI on the device.

[1247] The terminal transmits the user's adjustments to the server in real time, and the output is the adjustment data transmitted to the server.

[1248] The server receives the adjustments and generates a new design proposal, which is then sent back to the device.

[1249] The output is a revised interior design proposal.

[1250] Step 6: Budget review and closing

[1251] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user. The input is the furniture information sent from the server.

[1252] The terminal calculates the total budget and presents it to the user. The budget calculation is based on the price information of the selected furniture. The output is the total budget presented to the user.

[1253] The user confirms the budget and moves to the online payment screen. They enter the necessary payment information (credit card number, etc.) and make the payment.

[1254] The server receives the payment information and completes the payment process with the external payment provider. The output is a confirmation of successful payment.

[1255] Step 7: Delivery and assembly service

[1256] The server checks the stock status of the purchased furniture. The stock information is stored in a database and is searched using an SQL query. The input is the user's order information.

[1257] After the server checks the inventory, it automatically adjusts the delivery schedule. It works with the Delivery Administrator API to set the optimal delivery date. The output is the confirmed delivery date and time.

[1258] The server arranges delivery and assembly services, notifies the terminal of the confirmed delivery date and time, and outputs the delivery information notified to the user.

[1259] The user receives the furniture on the scheduled delivery date. The delivery staff arrives at the specified time and assembles and installs the purchased furniture. The output is a confirmation that assembly and installation have been completed.

[1260] (Application example 1)

[1261] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1262] While there are currently support systems that offer a complete range of services from interior design proposals to furniture purchases and installation, there are few ways for users to visually check the actual furniture placement and adjust it in real time. This often leads to problems such as the furniture not matching the image after purchase, making it difficult for users to design their home interiors without stress.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1264] In this invention, the server includes means for acquiring and uploading images of the user's interior, means for analyzing the characteristics of the interior from the uploaded images, means for displaying the generated interior design, means for the user to adjust the displayed design, means for calculating costs and making payments based on the selected furniture and design, means for delivering and assembling the selected furniture, and means for visually displaying the generated design in augmented reality using a smart device. This allows the user to visually check the actual furniture placement and adjust it in real time, easily achieving the optimal interior design.

[1265] "Means for acquiring and uploading images of one's own room" refers to a function that allows a user to take a photo of the room using a smartphone or camera and send the image data to the system.

[1266] "Means for analyzing the interior features from uploaded images" refers to a function in which the system analyzes uploaded image data and extracts features such as the interior dimensions, furniture arrangement, and color hue.

[1267] The "means for displaying the generated interior design" is a function for visually presenting to the user an interior design proposal generated based on the analyzed data.

[1268] "Means for users to adjust the displayed design" refers to a function that provides an interface that allows users to change the furniture placement, color, type, etc. of the presented design proposal.

[1269] "Means for calculating costs and making payments based on the selected furniture and design" is a function that calculates the total cost based on the furniture and design selected by the user and allows payment to be made online.

[1270] "Means for delivering and assembling selected furniture" is a function that arranges for the furniture purchased by the user to be delivered to the specified address and properly assembled and installed.

[1271] "Means for visually displaying the generated design in augmented reality using a smart device" refers to a function that uses devices such as smart glasses or a headset to display the generated interior design superimposed on the actual room, allowing the user to check its layout in real time.

[1272] This invention is a system that allows users to take pictures of their own rooms, propose and adjust interior designs based on the images, and ultimately purchase and install furniture efficiently. Specific embodiments of this system are described below.

[1273] First, the user takes a photo of their room using a device such as a smartphone or tablet. The photo is temporarily saved on the device and then sent to the server by pressing the upload button. At this time, the image data is transferred in an appropriate format.

[1274] The server then receives the uploaded images and uses an image analysis module to automatically analyze characteristics such as the room's dimensions, furniture placement, and color. Based on the analysis results, a generative AI model automatically generates interior design proposals. The output of the generative AI model is adjusted taking into account the user's past preference data and current interior design trends.

[1275] The generated interior design proposals are displayed in real time on smart devices. For example, if the user is wearing smart glasses, the generated design is overlaid on the real room using augmented reality (AR) technology, allowing the user to visually confirm the furniture and design as if they were actually placed in the room.

[1276] If users like the design, they can adjust the color, placement, and type of furniture. These adjustments are made through an interactive UI, with real-time feedback.

[1277] The terminal also displays detailed information about the selected furniture (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user checks the budget and, if there are no problems, makes an online payment. The payment process is handled by the server.

[1278] Finally, the server checks the inventory status of the purchased furniture and automatically schedules the delivery. It also arranges for delivery and assembly services and notifies the user of the delivery date and time. The user picks up the furniture on the scheduled delivery date, and the delivery staff assembles and sets it up.

[1279] For example, a user takes three photos of their living room and uploads them according to the app's instructions. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's smart glasses, and the user can review it and change the color of the sofa from red to blue and move the TV stand to the other side of the window. Finally, the user confirms the furniture price and total budget, and makes an online payment. The delivery staff delivers the furniture on the specified date and assembles it according to the user's wishes.

[1280] An example of input to a generative AI model is the following prompt:

[1281] "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m."

[1282] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1283] Step 1: The user takes a photo of their room using a smartphone or tablet.

[1284] Input: A photo of an interior taken with a smartphone or tablet.

[1285] Output: The captured photos are temporarily saved on the device.

[1286] Specific behavior: The user launches the camera app and takes several photos of the room. If necessary, the app will display guidance on how to take photos properly.

[1287] Step 2: The user uploads a photo to the server.

[1288] Input: The indoor photo taken in step 1.

[1289] Output: Photo data sent to the server by pressing the upload button.

[1290] How it works: The user clicks the upload button in the app, and the selected photos are sent to the server. The photo data is transferred in a standard format such as JPEG.

[1291] Step 3: The server analyzes the uploaded image.

[1292] Input: Photo data of the room sent to the server.

[1293] Output: Analyzed interior features such as dimensions, furniture layout, and color.

[1294] Specific operation: The server uses the image analysis module to analyze the dimensions, furniture arrangement, and color of the photo, and as a result, obtains the interior feature data.

[1295] Step 4: The server uses the generative AI model to generate interior design proposals.

[1296] Input: Analyzed indoor feature data, user's past preference data, current interior trends.

[1297] Output: Generated interior design proposals.

[1298] How it works: The server uses the generative AI model to generate interior design proposals based on a prompt such as "Generate a modern interior design for a living room based on the provided photos. Consider the current trends and user preferences. The room dimensions are 5m by 4m." The output design proposals are saved in a digital format.

[1299] Step 5: The device displays the generated interior design proposal to the user.

[1300] Input: Generated interior design proposal.

[1301] Output: Interior design proposal displayed on the user's device.

[1302] How it works: Smart glasses, tablets, and other devices receive the design proposals and use augmented reality (AR) technology to overlay them onto the user's field of vision, allowing the user to view the design proposals through their device.

[1303] Step 6: The user adjusts the design proposal.

[1304] Input: Adjustment instructions from the user (changes to furniture color, placement, type, etc.).

[1305] Output: Adjusted interior design proposal.

[1306] Specific operation: The user operates the device to change the color, arrangement, type, etc. of the furniture in the design proposal. The changes are reflected in real time, and the final design proposal is updated.

[1307] Step 7: The device will display detailed information about the selected furniture and calculate your budget.

[1308] Input: Adjusted interior design proposal.

[1309] Output: Furniture details (price, brand, dimensions, etc.), calculated budget.

[1310] Specific operation: The device displays detailed information about the selected furniture based on the adjusted design proposal. The total cost and individual costs are also calculated and presented to the user.

[1311] Step 8: The user makes an online payment.

[1312] Input: Final interior design proposal and adjusted budget.

[1313] Output: Payment completion notification.

[1314] What happens: The user presses the payment button on the device to complete the online payment. The payment is made through a secure protocol, and a notification is sent to the server upon success.

[1315] Step 9: The server arranges furniture delivery and assembly services.

[1316] Input: Payment completion notification, inventory information.

[1317] Output: Notification of delivery date and time and arrangement of assembly staff.

[1318] Specific operation: The server checks the furniture inventory and schedules the optimal delivery date and assembly service. The delivery date and time are notified to the user's device, and delivery and assembly work are carried out on the specified date.

[1319] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1320] This invention relates to a system that allows for efficient and easy home interior design, and in particular provides more user-friendly interior designs by combining an emotion engine that recognizes the user's emotions and reflects them in design proposals. The program for this system is specifically implemented as follows through interactions between a server, terminals, and users, and the use of the emotion engine.

[1321] Detailed explanation of functions

[1322] 1. Take and upload a photo

[1323] The user takes multiple photos using the camera function of their smartphone or tablet to capture various locations in their room. The photos are temporarily stored on the device and the user can select one.

[1324] The device provides an upload button, and when the user presses the upload button to upload the selected photo, the photo data is sent to the server. The photo data is transferred in an appropriate format.

[1325] 2. Interior design proposals

[1326] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[1327] The server uses the generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[1328] 3. Adjust the design

[1329] The device displays the generated interior design proposals to the user in an interactive UI.

[1330] The user can view the displayed design proposal and adjust the color, placement, type, etc. of the furniture. The device reflects the user's changes in real time and displays the updated design.

[1331] 4. Utilizing the Emotion Engine

[1332] To recognize the user's emotions, the device uses an emotion engine, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[1333] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[1334] 5. Budget confirmation and settlement

[1335] The device displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user.

[1336] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[1337] 6. Delivery and assembly services

[1338] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[1339] The user prepares to receive the furniture on the scheduled delivery date.

[1340] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[1341] Specific examples

[1342] The user takes three photos of their living room and follows the app's instructions to upload them.

[1343] After analyzing the photo, the server generates a modern interior design proposal, which is then displayed on the user's device.

[1344] The user changes the color of the sofa from red to blue and moves the TV stand to the other side of the window.

[1345] The device reads the user's facial expression, and the emotion engine recognizes that the user is in a relaxed state, and sends this to the server.

[1346] Based on the data from the emotion engine, the server will suggest more subdued colors and relaxing furniture.

[1347] The user checks the furniture price and total budget and makes an online payment. The server schedules the delivery and assembly service of the purchased furniture and notifies the terminal.

[1348] Delivery staff will deliver the furniture on the specified date and assemble and set it up according to the user's wishes.

[1349] As described above, this system provides users with the proposal and realization of highly convenient interior designs through a series of processes including the emotion engine.

[1350] The processing flow will be explained below.

[1351] Step 1:

[1352] Users take photos of their rooms using the camera function on their smartphones or tablets, and the photos are temporarily saved on the device.

[1353] Step 2:

[1354] The user selects the appropriate photo from the ones they have taken and presses the upload button in the app to send the photo data to the server. The device then transfers the photo data in the appropriate format.

[1355] Step 3:

[1356] The server launches an image analysis module to analyze the received photo data, which automatically extracts characteristics such as room dimensions, furniture placement, and color hues.

[1357] Step 4:

[1358] Based on the results of the image analysis module, the server uses a generative AI model to generate interior design proposals, taking into account the user's past preference data and current interior trends.

[1359] Step 5:

[1360] The server sends the generated interior design proposals to the device, which then displays the received design proposals to the user in an interactive UI.

[1361] Step 6:

[1362] The user can adjust the color, placement, type, etc. of the furniture while looking at the displayed design proposal. The device reflects the user's changes in real time and displays the updated design.

[1363] Step 7:

[1364] The device reads the user's facial expressions and tone of voice, and the emotion engine recognizes their current emotional state. For example, if the user is smiling, the emotion is determined to be "joy."

[1365] Step 8:

[1366] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[1367] Step 9:

[1368] The user then finalizes the design and moves on to the next step. The device displays detailed information about the selected furniture (price, brand, dimensions, etc.) and calculates the overall budget.

[1369] Step 10:

[1370] The user checks the displayed budget and furniture details and makes an online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[1371] Step 11:

[1372] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[1373] Step 12:

[1374] The terminal notifies the user of the delivery date and time, and the user prepares to receive the furniture on the scheduled delivery date.

[1375] Step 13:

[1376] The delivery staff will deliver the furniture to the user's address on the specified date and time and assemble it. The user can then check the assembled furniture and rate whether the service was satisfactory.

[1377] Example 2

[1378] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1379] Conventional interior design systems have difficulty proposing personalized designs that reflect the user's emotions. Furthermore, selecting and arranging furniture by oneself can be time-consuming, which can lead to a lack of completeness in the design. Furthermore, the complicated procedures involved in calculating costs and arranging furniture delivery and assembly reduce user convenience.

[1380] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing the characteristics of the space from the transmitted image, a means for automatically generating layout proposals using a generative AI model, and a means for recognizing the user's emotional state and adjusting the proposals based on that. This enables personalized interior design that reflects the user's emotions, and also enables efficient and consistent design adjustments, cost calculations, and furniture delivery and assembly procedures.

[1381] "Image of space" is video information taken by a user to show the situation of his / her specific location.

[1382] "Means of transmission" refers to the technology that transfers images of the space taken by the user to a server using wireless communication or the Internet.

[1383] "Spatial characteristics" refers to information extracted through image analysis, such as spatial dimensions, furniture arrangement, color, and lighting.

[1384] The "means of analysis" refers to technology that automatically extracts spatial characteristics using image analysis modules and artificial intelligence algorithms.

[1385] The "layout proposal" is a plan that proposes the optimal furniture layout and interior design based on the analyzed characteristics of the space.

[1386] The "display means" refers to a technology for visually presenting the generated layout plan to the user, and includes, for example, a mobile app or a web app.

[1387] "Adjustment means" refers to techniques that allow users to interactively change and adapt the displayed layout plan.

[1388] "Cost" refers to the total cost of the selected furniture and interior.

[1389] The "processing means" refers to the technology that calculates the cost of the furniture and interior items selected by the user and allows for online payment.

[1390] "Delivery and assembly means" refers to the technology and services for transporting the selected furniture to the customer's designated location and assembling it on-site.

[1391] A "generative AI model" is an artificial intelligence algorithm with natural language processing and image generation capabilities, and is a model for automatically generating placement plans.

[1392] "Emotional state" refers to the mental and emotional state of the user ascertained by analyzing facial expressions and tone of voice.

[1393] The "means of recognition" is a technology that uses an emotion engine to identify the user's emotional state.

[1394] This invention is a system that proposes personalized interior design taking into account the user's emotions. This system is realized using the interaction of a server, a terminal, and a user. Below, we will explain how data is processed using specific hardware and software.

[1395] First, a user takes multiple photos of their room using a device such as a smartphone or tablet. These photos are temporarily stored on the device, and the user can select one. The user then presses the upload button on the device to send the selected photos to the server. The server then saves the received photo data in an appropriate format (e.g., JPEG or PNG).

[1396] The server then launches an image analysis module to analyze the received photo data. Specifically, image processing libraries such as OpenCV and TensorFlow can be used. The analysis module automatically extracts characteristics such as room dimensions, furniture placement, and color hues. This extracted data is used to create interior design proposals.

[1397] After the analysis is complete, the server uses a generative AI model to automatically generate interior design proposals. Examples of such models include DALL-E and GPT-4. The design generation takes into account not only the analyzed room characteristic data, but also the user's past preference data and current interior trends. An example of a prompt sentence input to the generative AI model is as follows:

[1398] Example prompt sentence:

[1399] Room size: 5m x 6m. Furniture arrangement: Sofa, TV stand, coffee table. Preferred style: Modern. Favorite colors: Blue, gray. Current interior trends: Natural materials, simple design.

[1400] The generated interior design proposal is displayed to the user through the device's interactive user interface (UI). The user can interactively adjust the color, placement, and type of furniture in the displayed design proposal. For example, the user can change the color of a sofa from red to blue, or move a TV stand to the other side of a window. The device then reflects the user's changes in real time and redisplays the updated design proposal.

[1401] The device then uses an emotion engine to analyze the user's facial expressions and tone of voice. The emotion engine uses technologies such as the Emotion API from Microsoft Azure Cognitive Services. The emotional data obtained as a result of the analysis contains information such as whether the user is relaxed or excited. Based on this, the server can further adjust the design proposal. For example, if the user is in a relaxed state, the system will suggest calming colors and relaxing furniture.

[1402] Finally, the terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.), calculates the total budget, and presents it to the user. The user confirms the budget and, if there are no problems, makes an online payment. The payment information is sent from the terminal to the server, and the payment is processed. The server then arranges for delivery and assembly services, and notifies the terminal of the detailed delivery date and time.

[1403] The customer prepares to receive the furniture on the specified delivery date. The delivery staff delivers the furniture to the customer's address on the specified date and assembles it on site. The customer checks the completed furniture and evaluates whether the service was satisfactory.

[1404] Through the above process, the system provides users with an easy and efficient way to propose and realize interior designs. By utilizing the emotion engine, more personalized design proposals are provided, improving user satisfaction.

[1405] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1406] Step 1:

[1407] A user launches the camera app on their smartphone or tablet and takes a photo of their room. They take multiple photos of different rooms, such as the living room and bedroom. These photo data are temporarily stored on the device and the user can select one. The input is the captured photo data, and the output is the selected photo data.

[1408] Step 2:

[1409] The device displays a photo upload button, and the user presses the upload button to select a photo. The device converts the selected photo into an appropriate format (e.g., JPEG, PNG) and sends it to the server. The input is the selected photo data, and the output is the photo data sent to the server.

[1410] Step 3:

[1411] The server stores the received photo data in storage. Next, the server launches an image analysis module (e.g., OpenCV, TensorFlow) to analyze the photo data. The analysis module identifies the room's dimensions, furniture layout, color hue, etc. The input is the received photo data, and the output is the analyzed room characteristic data.

[1412] Step 4:

[1413] The server uses the analyzed room characteristic data to launch a generative AI model (e.g., DALL-E, GPT-4) and automatically generate interior design proposals. The generation process also takes into account the user's past preference data and current interior trends. The input is the analyzed room characteristic data and the user's preference data, and the output is the generated interior design proposals.

[1414] Step 5:

[1415] The device displays the generated interior design proposal to the user in an interactive user interface (UI), allowing the user to check the displayed design proposal. The input is the generated interior design proposal, and the output is the design proposal displayed to the user.

[1416] Step 6:

[1417] The user adjusts the design proposal using the UI. For example, the user may change the color of the sofa from red to blue or rearrange the furniture. The device reflects the user's changes in real time and redisplays the updated design proposal. The input is the user's adjustment data, and the output is the updated design proposal.

[1418] Step 7:

[1419] The device uses the built-in camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state. This is done using an emotion engine (e.g., Microsoft Azure's Emotion API). The input is the user's facial expression and voice data, and the output is the recognized emotional state data.

[1420] Step 8:

[1421] The server further adjusts the interior design proposal based on the emotional state data obtained from the emotion engine. For example, if the user is in a relaxed state, it will suggest calming colors and relaxing furniture. The input is the emotional state data, and the output is the adjusted interior design proposal.

[1422] Step 9:

[1423] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the selected furniture data, and the output is the calculated budget data.

[1424] Step 10:

[1425] The user checks the budget and, if there are no problems, presses the online payment button. The terminal sends the payment information to the server, which then processes the payment. The input is the payment information, and the output is payment completion data.

[1426] Step 11:

[1427] The server checks the inventory and sets the delivery date for the purchased furniture. The server arranges the delivery and assembly service and notifies the terminal of the details. The input is the purchased furniture data, and the output is the delivery schedule data.

[1428] Step 12:

[1429] The user receives the furniture on the specified delivery date. The delivery staff brings the furniture to the user's address and assembles it on site. The user checks the completed furniture and rates whether the service was satisfactory. The input is information about the delivery and assembly service, and the output is the user's satisfaction rating data.

[1430] (Application example 2)

[1431] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1432] Conventional interior design systems have the problem of reducing user satisfaction because they propose designs based solely on the physical characteristics of the room and past preference data, without taking the user's emotional state into consideration. In particular, the interior design of autonomous vehicles requires providing a design that is optimal for the user's emotional state while traveling, but this issue has not been adequately resolved.

[1433] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for taking and uploading a photo of the user's room, means for analyzing characteristics of the room from the uploaded photo, means for displaying generated interior design proposals, means for the user to adjust the generated interior design proposals, means for recognizing the user's emotional state, means for adjusting the interior design proposals based on the emotional data, means for calculating a budget based on the selected furniture and design and making payments, and means for delivering and assembling the selected furniture. This makes it possible to provide an interior design that is optimal for the user's emotional state in real time, thereby improving satisfaction.

[1434] "Means to take and upload photos of your room" is a function that allows you to take photos of your room or the inside of your car using a smartphone or camera and transfer the image data to the system's server.

[1435] "Means for analyzing room characteristics from uploaded photos" is a function for analyzing received photo data and extracting the dimensions, furniture arrangement, color tone, etc. of a room or vehicle interior.

[1436] The "means for displaying the generated interior design proposal" is a function for visually displaying the interior design proposal generated by the server on the user's terminal.

[1437] The "means for the user to adjust the generated interior design proposal" is a function that allows the user to change the color, layout, materials, etc. of the displayed interior design proposal.

[1438] The "means for recognizing the user's emotional state" is a function for analyzing the user's facial expression, tone of voice, etc., to identify the user's emotional state at that time.

[1439] The "means for adjusting interior design proposals based on emotional data" is a function for further optimizing interior design proposals based on the recognized emotional data.

[1440] "Means for calculating a budget and making payments based on the selected furniture and design" is a function that calculates the overall budget based on the furniture and design selected by the user and makes online payments within that budget.

[1441] The "means for delivering and assembling the selected furniture" is a function for delivering the determined furniture to the user and assembling the furniture.

[1442] "Means for automatically generating interior design proposals using a generative AI model based on the analyzed characteristics of a room" is a function that uses a generative AI model to automatically propose interior designs based on the analysis results of a room or car interior.

[1443] "Means for customizing interior design proposals based on the user's past preference data and current interior trends" is a function for customizing interior design proposals taking into account the user's past preference data and the latest interior trends.

[1444] This invention is a system for efficiently and easily customizing the interior design of an autonomous vehicle. In particular, it provides a more user-friendly interior design by combining an emotion engine that recognizes the user's emotional state and reflects it in the design proposal.

[1445] The system program is specifically implemented as follows through the interaction between the server, the terminal, and the user and the use of the emotion engine.

[1446] Taking and uploading photos

[1447] Users can take photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device and can be selected by the user.

[1448] The device provides an upload button to send the user-selected photos to the server, where the photo data is transferred in the appropriate format.

[1449] Interior design proposals

[1450] The server then launches an image analysis module to analyze the received photo data, automatically extracting characteristics such as room or vehicle interior dimensions, furniture placement, and color hues.

[1451] The server uses a generative AI model to generate interior design proposals based on the analysis results, taking into account the user's past preference data and current interior trends.

[1452] Design Adjustments

[1453] The device displays the generated interior design proposal to the user in an interactive UI. Using a smartphone or head-mounted display, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture. The device reflects the user's changes in real time and displays the updated design.

[1454] Utilizing the Emotion Engine

[1455] The device uses an emotion engine to recognize the user's emotions, which analyzes the user's facial expressions and tone of voice to identify their current emotional state.

[1456] The server further adjusts the interior design proposals based on the emotional data obtained from the emotion engine. For example, if the user feels like relaxing, it will suggest calming colors and relaxing furniture.

[1457] Budget confirmation and payment

[1458] The terminal displays detailed information about the furniture selected by the user (price, brand, dimensions, etc.) and calculates the total budget and presents it to the user.

[1459] The user checks the budget and, if there are no problems, proceeds with online payment. The payment details are sent from the terminal to the server, which then processes the payment.

[1460] Delivery and assembly services

[1461] The server checks the inventory status of the purchased furniture and schedules the delivery. The server then arranges for delivery and assembly services and notifies the terminal of the delivery date and time.

[1462] The user prepares to receive the furniture on the scheduled delivery date.

[1463] The delivery staff will deliver the furniture to the user's address at the specified date and time and assemble it. The user can then check the assembled furniture and rate the service as satisfactory.

[1464] Specific examples

[1465] The user takes three photos of the interior of the autonomous vehicle and uploads them as instructed by the app. The server analyzes the photos and generates a modern-style interior design proposal. The generated design proposal is displayed on the user's device. The user changes the color of the sofa from red to blue and moves the TV stand to the opposite side of the window. The device reads the user's facial expression and the emotion engine recognizes that the user is in a relaxed state, and sends this information to the server. Based on the data from the emotion engine, the server suggests more muted colors and more relaxing furniture. The user checks the price and total budget of the furniture and makes an online payment. The server arranges a delivery and assembly service schedule for the purchased furniture and notifies the device. The delivery staff delivers the furniture on the specified date and assembles and sets it up according to the user's wishes.

[1466] An example of a prompt for the generative AI model is, "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state." In this way, the system can provide an interior design that best suits the user's emotional state in real time, improving satisfaction.

[1467] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1468] Step 1:

[1469] The user takes photos of various locations inside the autonomous vehicle using the camera function of their smartphone or head-mounted display. The photos are temporarily stored on the device. The input is a photo of each location, and the output is the photo data stored on the device. At this stage, the user can take multiple photos as needed and select the appropriate photo according to the application's instructions.

[1470] Step 2:

[1471] The device provides an upload button and sends the photos selected by the user to the server. The input here is the photo data selected by the user, and the output is the photo data sent to the server. When the user presses the upload button, the device converts the photo data into an appropriate format and transfers it to the server.

[1472] Step 3:

[1473] The server launches an image analysis module to analyze the received photo data. The input is the photo data stored on the server, and the output is characteristics such as the dimensions of the room or car interior, furniture arrangement, and color hue. The server uses image analysis algorithms to extract these characteristics from the photo data and prepare it for further processing.

[1474] Step 4:

[1475] The server uses a generative AI model to generate interior design proposals based on the analysis results. The input is the analyzed characteristic data of the room or car interior, and the output is an interior design proposal. In particular, the user's past preference data and current interior trends are also taken into consideration. For example, a prompt sentence such as "Based on a photo of the car interior, please suggest a modern-style interior with relaxing colors. The user is currently in a relaxed state" is input into the generative AI model.

[1476] Step 5:

[1477] The device displays the generated interior design proposal to the user in an interactive UI. The input is the interior design proposal sent from the server, and the output is the design proposal displayed on the device. At this stage, the user can view the displayed design proposal and adjust the color, arrangement, and type of furniture.

[1478] Step 6:

[1479] The device uses an emotion engine to recognize the user's emotions. Here, the device analyzes the user's facial expressions and tone of voice to identify their current emotional state. The input is the user's audio and video data, and the output is the analyzed emotional state data.

[1480] Step 7:

[1481] The server further adjusts the interior design proposal based on the emotional data obtained from the emotion engine. The input is the user's emotional state data and the interior design proposal, and the output is the adjusted design proposal. For example, if the user feels like relaxing, the server will suggest calming colors and relaxing furniture.

[1482] Step 8:

[1483] The terminal displays detailed information (price, brand, dimensions, etc.) of the furniture selected by the user, calculates the total budget, and presents it to the user. The input is the adjusted interior design plan and detailed furniture information, and the output is the calculated budget and displayed information.

[1484] Step 9:

[1485] The user checks the budget and makes an online payment. The input is the calculated budget and the user's payment information, and the output is payment confirmation data. The payment details are sent from the terminal to the server, which then processes the payment.

[1486] Step 10:

[1487] The server checks the inventory status of the purchased furniture and sets a delivery date. The server also arranges for delivery and assembly services and notifies the terminal of the delivery date and time. The input is furniture inventory data and the user's desired delivery date and time, and the output is delivery and assembly schedule information.

[1488] Step 11:

[1489] The user prepares to receive the furniture on the scheduled delivery date. The delivery staff delivers the furniture to the user's address on the specified date and time and assembles it. The input is the schedule information and the furniture itself, and the output is a report that the assembly is complete. The user checks the assembled furniture and evaluates whether the service was satisfactory.

[1490] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1491] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1492] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1493] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1494] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1495] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1496] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1497] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1498] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1499] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1500] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1501] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1502] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1503] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1504] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1505] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1506] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1507] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1508] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1509] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1510] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1511] The following is further disclosed regarding the above embodiment.

[1512] (Claim 1)

[1513] A way to take and upload photos of your room,

[1514] A means of analyzing room characteristics from uploaded photos;

[1515] a means for displaying the generated interior design proposal;

[1516] a means for the user to adjust the displayed design proposal;

[1517] A means of calculating the budget and making payments based on the selected furniture and design;

[1518] means for delivering and assembling the selected furniture;

[1519] A system including:

[1520] (Claim 2)

[1521] 10. The system of claim 1, further comprising means for automatically generating interior design suggestions using a generative AI model based on the analyzed room characteristics.

[1522] (Claim 3)

[1523] 10. The system of claim 1, further comprising means for customizing interior design suggestions based on user historical preference data and current interior trends.

[1524] (Claim 4)

[1525] 10. The system of claim 1, further comprising means for automatically arranging a delivery date and time for the selected furniture and notifying the user.

[1526] "Example 1"

[1527] (Claim 1)

[1528] A means for users to take and upload photos of their rooms;

[1529] A means for the server to analyze room characteristics from the uploaded photograph;

[1530] A means for the server to automatically generate interior design proposals using a generative AI model based on the analyzed characteristics;

[1531] a means for displaying the generated interior design proposal on the terminal;

[1532] a means for the user to adjust the displayed design proposal; and

[1533] A means for the terminal to calculate a budget and make a payment based on the selected furniture and design;

[1534] a means for the server to deliver and assemble the selected furniture;

[1535] A system including:

[1536] (Claim 2)

[1537] 10. The system of claim 1, further comprising means for the server to customize the interior design suggestions based on the user's past preference data and current interior trends.

[1538] (Claim 3)

[1539] 10. The system of claim 1, wherein the terminal further comprises means for reflecting user adjustments to the interior design proposal in real time.

[1540] "Application Example 1"

[1541] (Claim 1)

[1542] A way to capture and upload images of your room,

[1543] A means for analyzing indoor features from uploaded images;

[1544] a means for displaying the generated interior design;

[1545] a means for the user to adjust the displayed design; and

[1546] A means for calculating and settling costs based on the selected furniture and design;

[1547] means for delivering and assembling the selected furniture;

[1548] a means for visually displaying the generated design in augmented reality using a smart device;

[1549] A system including:

[1550] (Claim 2)

[1551] 10. The system of claim 1, further comprising means for automatically generating an interior design using a generative AI model based on the analyzed interior features.

[1552] (Claim 3)

[1553] 10. The system of claim 1, further comprising means for customizing an interior design based on a user's historical preference data and current interior trends.

[1554] "Example 2: Combining Emotion Engines"

[1555] (Claim 1)

[1556] A means to take and send images of your space;

[1557] means for analyzing spatial characteristics from the transmitted image;

[1558] a means for displaying the generated placement plan;

[1559] a means for the user to adjust the displayed placement plan;

[1560] a means for calculating and processing costs based on the selected furnishings and designs;

[1561] a means of delivering and assembling the selected furnishings;

[1562] a means for recognizing a user's emotional state and adjusting the proposed suggestions accordingly;

[1563] A system including:

[1564] (Claim 2)

[1565] 10. The system of claim 1, further comprising means for automatically generating placement proposals using a generative AI model based on characteristics of the analyzed space.

[1566] (Claim 3)

[1567] 10. The system of claim 1, further comprising: means for customizing placement suggestions based on user historical preference data and current placement trends.

[1568] "Application example 2 when combining emotion engines"

[1569] (Claim 1)

[1570] A way to take and upload photos of your room,

[1571] A means of analyzing room characteristics from uploaded photos;

[1572] a means for displaying the generated interior design proposal;

[1573] A means for the user to adjust the generated interior design proposal;

[1574] means for recognizing the emotional state of a user;

[1575] a means for adjusting the interior design proposal based on the emotion data; and

[1576] A means of calculating the budget and making payments based on the selected furniture and design;

[1577] means for delivering and assembling the selected furniture;

[1578] A system including:

[1579] (Claim 2)

[1580] 10. The system of claim 1, further comprising means for automatically generating interior design suggestions using a generative AI model based on the analyzed room characteristics.

[1581] (Claim 3)

[1582] 10. The system of claim 1, further comprising means for customizing interior design suggestions based on user historical preference data and current interior trends. [Explanation of symbols]

[1583] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 S...

Claims

1. A way to take and upload photos of your room, A means of analyzing room characteristics from uploaded photos; a means for displaying the generated interior design proposal; a means for the user to adjust the displayed design proposal; A means of calculating the budget and making payments based on the selected furniture and design; means for delivering and assembling the selected furniture; A system including:

2. The system of claim 1 , further comprising means for automatically generating interior design proposals using a generative AI model based on the analyzed room characteristics.

3. The system of claim 1 , further comprising means for customizing interior design suggestions based on the user's historical preference data and current interior trends.

4. The system of claim 1 further comprising means for automatically arranging a delivery date and time for the selected furniture and notifying the user.

Images